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ABSTRACT 


Human  error  has  been  identified  as  a  factor  in  virtually  every  major  maritime 
mishap  over  the  past  decade.  The  Department  of  Defense  (DoD)  currently 
employs  the  Human  Factors  Analysis  and  Classification  System  (HFACS) 
taxonomy  to  identify  and  quantify  human  error  in  major  mishaps.  HFACS  divides 
errors  into  categories,  sub-codes,  and  nano-codes.  The  generic  nature  of  DoD 
HFACS  raises  the  question  of  whether  or  not  a  domain-specific  version  for  the 
surface  Navy  could  be  applied  more  consistently.  Twenty-eight  subjects  (14 
Surface  Warfare  Officers  (SWOs)  and  14  non-SWOs)  employed  either  DoD 
HFACS  or  a  developmental  maritime  domain  specific  version,  HFACS-M,  to 
classify  findings  in  a  National  Transportation  Safety  Board  (NTSB)  maritime 
accident  investigation.  Fleiss’  Kappa  was  used  to  determine  inter-rater  reliability 
among  subjects.  The  results  of  this  study  revealed  that  SWOs  using  HFACS-M 
had  a  higher  inter-rater  reliability  (10.9%,  7.3%,  and  6.5%)  at  every  classification 
level  than  non-SWOs.  HFACS-M  itself  was  also  shown  to  have  a  slightly  higher 
overall  inter-rater  reliability  (5.7%,  7.4%,  and  3.6%)  than  DoD  HFACS.  The 
research  concluded  that  although  HFACS-M  performed  well,  further  testing  is 
necessary  to  validate  it. 
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EXECUTIVE  SUMMARY 


An  analysis  of  accident  investigations  throughout  the  surface  Navy  suggests  that 
nearly  every  mishap  contains  some  level  of  human  error.  To  identify  mishaps 
properly  for  mitigation  and  elimination,  the  Navy  must  have  an  effective  error 
classification  system.  The  Department  of  Defense  (DoD)  has  implemented  the 
Human  Factors  Analysis  and  Classification  System  (HFACS)  to  address  this  very 
issue.  HFACS  asserts  that  errors  arise  in  four  distinct  categories:  organizational 
influences,  supervision,  existing  preconditions,  or  the  very  acts  themselves.  Each 
category  is  divided  into  sub-codes,  and  each  sub-code  into  nano-codes  to 
identify  specific  errors.  HFACS  was  originally  developed  for  naval  aviation  but 
has  been  adapted  for  use  in  all  branches  of  service.  Several  published  studies 
suggest  that  domain-specific  error  classification  systems  may  lead  to  higher 
inter-rater  reliability.  To  this  end,  a  maritime  specific  version  of  HFACS,  HFACS- 
M,  was  developed. 

Twenty-eight  students  from  the  Naval  Postgraduate  School  (14  Surface 
Warfare  Officers  (SWOs)  and  14  non-SWOs)  received  training  on  either  DoD 
HFACS  or  HFACS-M  and  then  were  asked  to  employ  them  in  a  real-world 
scenario.  Subjects  were  asked  to  classify  1 1  findings  in  a  National  Transportation 
Safety  Board  maritime  accident  investigation  using  one  of  the  taxonomies  to 
assign  an  appropriate  nano-code.  The  subjects’  responses  were  compiled  into 
two  tables,  one  for  HFACS,  and  one  for  HFACS-M.  The  tables  were  then 
separated  between  SWOs  and  non-SWOs.  Inter-rater  reliability  was  calculated 
for  each  error  classification  taxonomy  using  Fleiss’  Kappa.  Overall  inter-rater 
reliability  and  inter-rater  reliability  for  SWOs  and  non-SWOs  were  calculated. 
This  process  was  repeated  at  the  sub-code  and  category  level. 

Analysis  showed  that,  of  the  two  taxonomies,  HFACS-M  had  a  slightly 
higher  overall  inter-rater  reliability  at  every  level  (5.7%,  7.4%,  and  2.8%)  than 
DoD  HFACS.  When  using  the  domain-specific  taxonomy,  SWOs  displayed  a 


xv 


higher  inter-rater  reliability  (10.9%,  7.4%,  and  6.5%)  than  non-SWOs.  Non-SWOs 
did,  however,  have  a  slightly  higher  inter-rater  reliability  (10.2%,  4.3%,  and  8.4%) 
when  employing  DoD  HFACS. 

The  research  concluded  that,  in  this  particular  study,  SWOs  performed 
slightly  better  at  every  level  of  analysis  than  non-SWOs  when  applying  the 
domain-specific  error  classification  taxonomy.  It  was  also  found  that  HFACS-M 
had  a  slightly  higher  overall  inter-rater  reliability  at  each  level  than  DoD  HFACS. 
Due  to  a  small  sample  size  and  lack  of  trained  raters,  it  cannot  be  stated 
conclusively  that  HFACS-M  is  a  significantly  better  method  for  classifying  error  in 
the  surface  Navy.  It  can  be  concluded,  however,  that  the  results  of  this  study 
support  the  need  for  further  research.  Additionally,  the  Navy  should  attempt  to 
address  the  gaps  in  latent  distal  errors  and  maintenance-specific  errors. 
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I.  INTRODUCTION 


A.  OVERVIEW 

Human  error  has  been  a  cause  in  virtually  every  significant  mishap  within 
the  surface  Navy  for  the  past  several  decades.  Based  on  Naval  Safety  Center 
data  from  January  1992  through  December  1996,  human  error  was  found  to  be  a 
factor  in  100%  of  all  recorded  incidents  (Lacy,  1998).  As  such,  the  reduction  of 
human  error  has  been  a  key  focus  of  the  Navy,  as  well  as  other  organizations  for 
many  years. 

Reason’s  research  into  human  error  brought  him  to  the  belief  that  in  a 
perfect  world,  mishaps  are  nearly  always  preventable.  He  saw  each  accident  as 
an  event  that  could  be  prevented  at  different  points.  Much  like  slices  of  Swiss 
cheese,  these  layers  were  filled  with  holes  (Figure  1)  in  the  real  world.  Reason 
asserted  that  these  holes  were  due  to  some  combination  of  latent  and  active 
failures  (Reason,  1997). 


Some  ‘holes’ 
due  to  active 
\  failures 


Defences 
in  depth 


Other  ‘holes’'' 
due  to  latent 
conditions  / 


Figure  1 .  Reason’s  original  “Swiss  Cheese”  model 

(From  Reason,  1997) 


Reason’s  theory  was  a  catalyst  for  the  team  of  Shappell  and  Wiegmann, 
who  took  the  basics  of  the  theory  and  developed  a  method  for  attributing 
causality  in  accidents  (Shappell  &  Wiegmann,  2001).  The  Department  of  Defense 
(DoD)  Human  Factors  Analysis  and  Classification  System  (HFACS)  is  a 
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taxonomy  for  classifying  mishaps.  Using  the  “Swiss  cheese”  model  as  a  starting 
point,  Shappell  and  Wiegmann  assigned  names  to  each  of  the  layers,  or  levels 
(Figure  2).  DoD  HFACS  consists  of  four  levels:  organizational  influences, 
supervision,  preconditions,  and  acts;  the  holes  within  each  of  which  lead  to  the 
eventual  mishap.  At  each  level,  the  taxonomy  is  broken  down  into  categories,  or 
sub-codes,  and  then  into  nano-codes  (Shappell  &  Wiegmann,  2001).  The  surface 
Navy  currently  uses  DoD  HFACS  in  classifying  all  its  major  mishaps  (Department 
of  Defense,  2005). 


Mishap 


Figure  2.  The  “Swiss  Cheese”  model — HFACS  version 

(After  Reason,  1990;  DoD,  2005) 


Since  its  creation,  HFACS  has  been  widely  researched,  with  more  than 
90  articles  published  on  the  subject.  The  research  surrounding  HFACS  is 
effectively  split  into  two  categories,  DoD  HFACS  and  hybrid  versions  of  DoD 
HFACS.  Next,  the  research  is  further  broken  down  into  analysis  using  the 
HFACS  sub-codes  and  analysis  using  nano-codes.  Of  these  four  possible 
combinations,  the  most  prevalent  research  concerns  DoD  HFACS  at  the  sub¬ 
code  level,  while  the  least  common  examines  non-DoD  HFACS  at  the  nano-code 
level. 
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The  majority  of  HFACS  research  presupposes  the  mishap  ratings  are 
accurate.  Many  studies  use  a  consensus  method  whereby  a  group  of  experts 
discusses  the  factors  of  the  mishap  before  arriving  at  a  final  decision,  much  like 
what  would  occur  at  a  mishap  investigation  board.  Coding  at  the  categorical  level 
has  been  shown  to  have  less  inter-rater  error,  presumably  due  to  the  small 
number  of  sub-codes  (19)  compared  to  the  large  number  of  nano-codes  (144). 

Not  all  researchers  presuppose  sufficient  inter-rater  reliability,  however. 
O’Connor  has  published  several  papers  testing  the  reliability,  utility,  and  validity 
of  HFACS  using  trained  raters,  simulated  mishap  boards,  and  experienced 
aviators.  O’Connor’s  findings  suggest  the  need  for  more  robust  HFACS  training, 
particularly  for  end  users,  and  a  more  robust  verification  and  validation  process 
for  the  evaluation  system  being  used — HFACS  or  otherwise  (O’Connor,  2008; 
O’Connor,  Walliser,  &  Philips,  2010;  O’Connor  &  Walker,  201 1 ). 

Salmon,  Cornelissen,  and  Trotter  (2012)  also  questioned  HFACS’ 
reliability.  The  researchers  conducted  a  comparison  of  several  accident  analysis 
methods,  including  Accimap,  HFACS,  and  STAMP.  Although  they  concluded  that 
HFACS  was  a  better  system  to  use  in  a  large  organization,  such  as  the  DoD, 
they  raised  questions  about  HFACS’  reliability  and  were  concerned  about  the 
lack  of  domain  specificity  outside  of  aviation. 

Finally,  in  one  of  the  most  recent  studies  utilizing  HFACS,  Griggs  (2012) 
investigated  mishaps  within  the  commercial  maritime  sector  and  applied  HFACS 
to  a  series  of  48  mishaps.  His  research  determined  that,  “in  order  to  improve  the 
reliability  of  HFACS,  the  taxonomy  needs  to  be  relevant  to  the  maritime 
community”  (Griggs,  2012,  p.  85). 

B.  BACKGROUND 

Accidents  are  an  unfortunate  reality  within  the  United  States  (U.S.)  Navy, 
and  repair  funds  are  allotted  each  year  to  cover  the  costs.  Unfortunately,  as 
technology  advances,  the  cost  to  repair  systems  involved  in  these  mishaps 
increases  exponentially. 
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Failure  to  learn  from  past  mishaps  all  but  ensures  that  those  mishaps  will 
be  repeated  in  time.  To  identify  and  prevent  the  root  cause  of  hazards  that  result 
in  major  mishaps  properly,  the  Navy  convenes  safety  investigation  boards  (SIB) 
for  each  of  the  following: 

1.  All  on-duty  Class  A  mishaps  on  or  off  a  government 
installation  (while  performing  official  duties);  in 
commissioned  and  pre-commissioned  U.S.  Navy  ships  after 
delivery;  United  States  Naval  Ships  (USNS)  with  federal 
civilian  mariner  crews  in  the  Military  Sealift  Command 
(MSC);  Navy-owned  experimental  and  small  craft;  and  the 
ship's  embarked  equipment,  boats,  and  landing  craft,  or 
leased  boats. 

2.  Military  death  that  occurs  during  or  as  the  result  of  a  medical 
event  that  occurs  within  one  hour  after  completion  of  any 
command-directed  remedial  physical  training  (PT),  physical 
readiness  test  (PRT),  physical  fitness  testing  (PFT),  physical 
fitness  assessment  (PFA)  or  command-sponsored  activity 
during  normal  working  hours  regardless  of  any  pre-existing 
medical  condition. 

3.  On-duty  injury  where  death  or  permanent  total  disability  is 
likely  to  occur,  or  where  damage  estimates  may  be  expected 
to  exceed  one  million  dollars. 

4.  Hospitalization,  beyond  observation,  of  three  or  more 
personnel,  at  least  one  of  who  is  a  DoD  civilian,  involved  in  a 
single  mishap. 

5.  All  explosives  mishaps,  all  ordnance  impacting  off  range  and 
all  live  fire  mishaps  resulting  in  an  injury. 

6.  Any  mishap  that  a  controlling  command  (as  defined  in 
paragraph  1005.6)  determines  requires  a  more  thorough 
investigation  and  report,  beyond  that  provided  by  a 
command’s  safety  investigator.  (Department  of  the  Navy, 
p.  6-1 ,  2005) 

Upon  concluding,  each  SIB  produces  a  list  of  findings  and  follow-on 
recommendations.  The  SIB  analyzes  these  findings  to  determine  which  hazards 
were  causal  to  the  mishap,  and  which  were  contributory  (did  not  directly  cause 
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the  incident).  The  SIB  then  converts  the  causal  and  contributory  factors  to  nano¬ 
codes  using  HFACS  (Department  of  the  Navy,  p.  A-15,  2005). 

The  instruction  that  governs  the  SIB  process  provides  guidance  with 
respect  to  the  board’s  composition.  The  composition  is  required  to  be  as  follows: 

1.  Minimum  composition  of  an  SIB  is  three  members;  however, 
five  is  preferred. 

2.  The  appointing  authority  and  senior  member  of  the  board 
can  confer  and  agree  on  board  appointees  based  on  the 
type  and  severity  of  the  mishap. 

3.  For  afloat  mishaps,  all  members  must  be  commissioned 
Officers.  If  the  mishap  involves  more  than  one  naval 
command,  a  Navy,  Marine,  or  MSC  representative  as 
appropriate,  shall  be  a  member  of  the  SIB. 

4.  The  senior  member  appointed  to  the  SIB  shall  not  be  from 
mishap  command.  All  SIBs  shall  consist  of: 

a.  A  senior  member,  who  shall  be  a  commissioned 
Officer  (0-5  or  above),  a  senior  civilian  (GS-13  or 
higher),  or  a  senior  official  in  MSC  as  appropriate. 

(1)  A  military  senior  member  of  a  Navy  SIB  shall 
be  senior  to  the  commanding  officer  of  the 
command  or  unit  involved  in  the  mishap. 

(2)  The  senior  member  of  a  Marine  Corps  SIB 
shall  be  a  Marine  Corps  officer  or  a  senior 
civilian  (GS-13  or  higher),  and  shall  be  equal  to 
or  senior  in  grade  to  the  commander  of  the 
mishap  unit. 

(3)  In  cases  where  the  senior  member  requirement 
cannot  be  met,  the  appointing  authority  shall 
request  a  waiver  from  the  appropriate 
controlling  command. 

b.  At  least  two  additional  members  (one  of  whom  could 
be  a  subject  matter  expert  (SME)  on  equipment, 
systems  or  procedures).  (DON,  p.  6-3,  2005). 
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These  requirements  present  several  potential  issues.  First,  none  of  the  members 
is  required  to  have  any  background  or  training  in  HFACS  or  investigative 
procedures  (Department  of  the  Navy,  p.  6-3,  2005).  This  board  composition 
policy  creates  the  potential  for  incorrect  HFACS  coding.  Secondly,  HFACS,  now 
called  DoD  HFACS,  is  used  throughout  all  branches  of  military  service  and 
contains  generic  and  non-domain  specific  codes,  which  leads  to  the  greater 
likelihood  of  erroneous  coding. 

C.  PROBLEM  STATEMENT 

The  HFACS  taxonomy  converts  qualitative  mishap  data  to  categorical 
data  for  the  purpose  of  analysis.  The  results  of  these  analyses  are  used  to  help 
decision  makers  determine  how  money  should  be  spent  to  prevent  future 
mishaps.  If  a  mishap  is  coded  incorrectly,  that  information  is  entered  into  a 
database  and  could  lead  to  incorrect  assumptions  when  analyzed.  Given  the  low 
inter-rater  reliability  found  in  several  studies  using  DoD  HFACS  (as  low  as  36% 
overall  and  as  low  as  22.5%  for  causal  factor  agreement),  it  is  imperative  that  the 
reasons  for  this  disparity  be  investigated,  and  methods  to  improve  reliability  be 
explored  (Baysari,  Caponecchia,  McIntosh,  &  Wilson,  2009;  O’Connor,  2008; 
O’Connor  et  al. ,  2010;  O’Connor  et  al.,  201 1 ). 

This  study  seeks  to  gain  insight  into  the  existing  claims  of  sub-optimal 
inter-rater  reliability  when  using  HFACS  (Baysari  et  al.,  2009;  O’Connor,  2008; 
O’Connor  et  al.,  2010;  O’Connor  et  al.,  2011).  To  inform  decision  makers 
correctly  about  where  to  spend  tax  dollars,  mishap  coding  must  be  accurate.  This 
study  also  introduces  a  maritime-specific  version  of  HFACS  for  use  in  the  surface 
Navy,  referred  to  as  HFACS-M  (maritime),  in  an  effort  to  observe  whether  or  not 
a  domain-specific  version  of  HFACS  results  in  increased  inter-rater  reliability. 
The  study  also  considers  the  role  of  training  in  HFACS  coding. 

D.  OBJECTIVES 

The  purpose  of  this  thesis  is  to  compare  the  inter-rater  reliability,  usability, 

and  validity  of  HFACS  and  HFACS-M,  which  is  a  modification  to  HFACS 
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developed  by  the  author  and  tailored  specifically  to  surface  ship  mishaps.  The 
objective  is  to  use  the  results  to  identify  any  possible  gaps  in  the  human  error 
taxonomies  for  the  surface  Navy.  The  results  will  lead  to  updated  taxonomies  to 
ensure  that  the  U.S.  Navy  is  able  to  identify  human  error  correctly  and  reduce  the 
number  of  mishaps  in  the  future. 

E.  RESEARCH  QUESTIONS 

To  identify  potential  gaps,  overlaps,  and  errors  within  HFACS  and 
HFACS-M,  this  study  attempts  to  answer  the  following  research  questions. 

•  Do  Surface  Warfare  Officers  (SWOs)  and  Non-SWOs  show  the 
same  consistency  when  applying  DoD  HFACS? 

•  What  errors,  overlaps,  or  gaps,  if  any,  currently  exist  in  DoD 
HFACS? 

•  Does  a  tailored  version  of  HFACS  result  in  increased  inter-rater 
reliability  when  classifying  mishaps  within  the  surface  Navy?  Why 
or  why  not? 

F.  SCOPE  AND  LIMITATIONS 

This  research  was  limited  to  the  results  of  the  statistical  analysis  of  the 
data  collected  from  two  case  studies.  Although  this  research  focused  on  accident 
analysis  within  the  surface  Navy,  Naval  Safety  Center  data  for  major  afloat 
mishaps  was  restricted.  This  research  focused  on  the  HFACS  classification  of 
Class  A  Mishaps  as  defined  by  the  current  version  of  OPNAVINST  5102,  the 
Navy  and  Marine  Corps  Mishap  and  Safety  Investigation,  Reporting,  and  Record 
Keeping  Manual  (Department  of  the  Navy,  2005). 

G.  HSI 

This  section  discusses  the  applicable  domains  of  HSI  which  pertain  to  this 
research.  More  specifically,  the  areas  of  Manpower,  Personnel  and  Training 
(MPT)  and  Human  Factors  Engineering  (HFE)  are  considered  in  this  thesis. 
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1.  Manpower,  Personnel,  and  Training 

The  manpower  domain  of  HSI  seeks  to  develop  systems  that  “optimize 
manpower  and  keep  human  resource  costs  at  affordable  levels”  (DAU,  2009).  An 
example  of  a  manpower  issue  is  determining  the  optimal  number  of  sonar 
technicians  required  onboard  a  Guided  Missile  Destroyer  (DDG)  to  fill  three 
watch  sections.  Manpower  is  an  important  factor  in  mishap  investigation.  Many 
times  human  error  occurs  because  Sailors  are  overworked  or  severely  stressed. 
Overwork  in  military  settings  can  often  be  attributed  to  the  improper  manning  of  a 
system.  Improper  manning  has  been  shown  to  lead  directly  to  an  increase  in 
safety  related  mishaps  (Lazzaretti,  2008). 

The  personnel  domain  of  HSI  differs  from  manpower  in  that  it  focuses  on 
“human  aptitudes  (i.e.,  cognitive,  physical,  and  sensory  capabilities),  knowledge, 
skills,  abilities,  and  experience  levels  that  are  needed  to  properly  perform  job 
tasks”  (Defense  Acquisition  University,  2009).  From  a  human  error  perspective, 
the  selection  of  Sailors  and  Officers  with  inappropriate  qualifications  and 
experience  levels  is  tantamount  to  ensuring  a  mishap  will  occur  in  due  time. 

The  DAU  defines  training  as  “any  activity  that  results  in  enabling  users, 
operators,  maintainers,  leaders  and  support  personnel,  to  acquire,  gain  or 
enhance  knowledge,  skills,  and  concurrently  develops  their  cognitive,  physical, 
sensory,  team  dynamics  and  adaptive  abilities  to  conduct  joint  operations  and 
achieve  maximized  and  fiscally  sustainable  system  life  cycles”  (Defense 
Acquisition  University,  2009).  As  systems  employ  more  technology,  the  number 
of  personnel  needed  to  operate,  maintain,  and  support  the  system  should 
decrease.  To  balance  this,  however,  more  training  is  required.  In  the  surface 
Navy,  command  wide,  departmental,  and  divisional  training  provide  invaluable 
knowledge  to  shipboard  personnel.  Failure  to  provide  specific  training  leads  to 
human  error,  which  leads  to  mishaps. 
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2.  Human  Factors  Engineering 

HFE  is  the  HSI  domain  that  supports  many  of  the  other  domains.  HFE 
seeks  to  ensure  systems  “capitalize  on  and  do  not  exceed  the  abilities  (cognitive, 
physical,  sensory,  and  team  dynamic)  of  the  user  population”  (Defense 
Acquisition  University,  2009).  In  systems  that  have  had  HFE  applied  properly 
during  the  design  process,  a  significant  reduction  often  occurs  of  either  cognitive 
or  physical  workload,  or  both.  Consequently,  failing  to  apply  proper  HFE  during 
system  development  can  be  the  cause  of  mishaps  due  to  physical  or  cognitive 
overload  of  the  human. 

H.  ORGANIZATION 

This  thesis  is  divided  into  six  chapters.  Chapter  I  provides  a  synopsis  of 
human  error  research  and  some  background  on  the  development  and  uses  of 
HFACS.  Chapter  II  provides  a  review  of  the  available  research  on  HFACS. 
Chapter  III  explains  how  the  HFACS-M  taxonomy  was  developed  and  the 
methodology  used  to  evaluate  HFACS  and  HFACS-M.  Chapter  IV  provides  an 
analysis  of  the  resulting  data,  and  addresses  the  significant  issues  uncovered  by 
the  research.  Chapter  V  discusses  the  implications  of  the  study’s  results.  Chapter 
VI  offers  conclusions  and  recommendations  for  future  research. 


9 


THIS  PAGE  INTENTIONALLY  LEFT  BLANK 


10 


II.  LITERATURE  REVIEW 


A.  MISHAPS 

Mishaps  comprise  the  largest  unintended  costs  for  the  surface  Navy 
today.  In  times  of  financial  constraints,  the  Navy  cannot  afford  to  spend  tax 
dollars  on  incidents  that  should  not  have  occurred,  given  proper  planning, 
training,  and  preparation.  Mishaps,  however,  are  an  inevitable  part  of  any 
organization.  As  Reason  noted,  organizational  accidents  are  “comparatively  rare, 
but  often  catastrophic,  events  that  occur  within  complex  modern  technologies” 
(Reason,  1997,  p.  1 ). 

Reason  explained  his  theory  of  how  mishaps  occur  using  the  terms 
hazards,  defenses,  and  losses  (Figure  3). 


Figure  3.  Relationship  between  hazards,  defenses,  and  losses 

(From  Reason,  1997) 


A  hazard  is  a  potential  mishap  or  something  that  could  go  wrong  if  not 
prevented.  In  the  surface  Navy,  such  a  hazard  is  a  ship  running  aground,  or 
colliding  with  another  ship.  Losses  are  the  result  of  an  undefended  or 
unprevented  hazard.  Losses  come  in  the  form  of  injury  or  loss  of  life  to  personnel 
or  damage  to  equipment. 

Defenses,  on  their  most  basic  level,  are  created  to  prevent  losses  and,  as 
Reason  explains,  serve  one  or  more  specific  functions.  First,  they  “create  an 
understanding  and  awareness  of  local  hazards”  (Reason,  1997,  p.  7).  In  the 
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surface  Navy,  these  defenses  could  be  a  Local  Notice  to  Mariners  report,  Coast 
Guard  broadcast  or  warning,  or  even  a  Naval  Criminal  Investigative  Service 
(NCIS)  port  brief  before  sailors  go  ashore.  Second,  they  provide  guidance  on 
safe  operation  that  could  be  a  Navy  standard  operating  procedure  (SOP), 
maintenance  requirement  card  (MRC),  or  a  safety  checklist.  Third,  Reason 
asserts  that  defenses  also  “provide  alarms  and  warnings  when  danger  is 
imminent”  (Reason,  1997,  p.  7)  that  include  tank  low-level  alarms,  smoke 
detection  and  heat  detection  sensors,  and  chemical  detection  units  on 
engineering  equipment.  Fourth,  these  defenses  will  return  the  system  to  a  normal 
operating  state  following  an  emergency,  which  includes  releasing  fire  zone  doors 
following  a  fire,  or  recycling  vent  dampers  following  a  missile  launch.  Defenses 
also  act  as  barriers,  primarily  physical  ones,  to  prevent  the  loss  from  actually 
occurring  or  to  mitigate  it.  In  the  fleet,  these  types  of  defenses  could  be  a  firing 
cutout,  which  prevents  the  system  from  engaging  the  superstructure  of  a  ship,  or 
any  redundant  safety  measure  to  prevent  spills  of  chemicals  or  fuels.  In  some 
situations,  defenses  are  needed  “to  contain  and  eliminate  the  hazards  should 
they  escape  this  barrier”  (Reason,  1997,  p.  7),  which  can  be  a  floating  oil  barrier 
placed  around  a  ship  when  it  pulls  alongside  a  pier,  or  an  agent,  such  as  Halon 
or  aqueous  film  forming  foam  (AFFF),  which  are  designed  to  eliminate  or  contain 
fires.  Finally,  defenses  provide  a  way  to  exit  an  area  or  save  human  lives  in  the 
event  the  primary  and  secondary  barriers  fail  (Reason,  1997)  that  can  be 
implemented  through  escape  trunks  or  scuttles,  first  aid  or  eyewash  stations,  life 
rafts,  and  distress  beacons. 

Defenses,  however,  are  not  perfect  in  practice.  Defenses  are  often 
operated  by  humans  who  are  prone  to  error.  Additionally,  many  defenses  require 
some  amount  of  warning  time  to  be  fully  activated  or  effective.  To  this  end, 
Reason  developed  the  concept  of  the  “Swiss  Cheese  Model”  (Reason,  1997). 

Figure  1  shows  the  basic  concept  of  the  Swiss  cheese  model,  which 
follows  from  his  initial  established  relationship  between,  hazards,  defenses,  and 
losses.  Defenses  (Swiss  cheese)  have  holes  resulting  from  active  or  latent 
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failures  in  the  defenses.  As  Reason  explains,  accident  causation  is  dynamic,  and 
can  be  triggered  locally,  occur  from  defects  in  the  defenses  themselves,  or  be 
caused  by  atypical  conditions  (Reason,  1990). 

Reason  developed  a  model  to  explain  how  the  hazard  to  loss  process 
worked  in  relation  to  latent  and  active  failures  (Figure  4).  The  triangle  portion  of 
the  figure  represents  the  factors  or  conditions  leading  up  to  an  event 
(represented  by  the  rectangle  at  the  top).  Latent  or  active,  these  failures  work 
together  to  create  an  error  chain  that  eventually  resulted  in  a  loss  (Reason, 
1997). 


Figure  4.  Stages  in  the  development  and  investigation  of  an  organizational 

accident  (From  Reason,  1997,  p.  17) 

As  an  example,  a  ship  has  been  extended  on  deployment,  which  has 
taken  its  toll  on  the  crew  (latent,  organizational  factor).  The  helmsman  has  had 
insufficient  sleep  (latent,  local  workplace  factor)  as  he  steers  the  ship  late  at 
night.  The  Conning  Officer  is  supposed  to  be  watching  the  navigation  situation, 
but  is  preoccupied  by  his  upcoming  Surface  Warfare  Officer  (SWO)  board,  and  is 
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not  paying  attention  (active,  local  workplace  factor).  The  ship  is  transiting  a 
narrow  channel  and  strays  into  shoal  water  on  its  starboard  side  due  to  the 
helmsman  nodding  off  with  no  one  paying  attention  to  him  (active,  unsafe  act). 
The  Officer  of  the  Deck  (OOD)  and  Junior  Officer  of  the  Deck  (JOOD)  stand  out 
on  the  port  bridge  wing  and  have  a  discussion  about  the  NCAA  Basketball 
Tournament  currently  going  on  (active,  local  workplace  factor).  The  resulting  loss 
is  a  grounding  of  the  ship,  millions  of  dollars  in  repairs,  and  the  end  of  several 
careers. 

B.  ACCIDENT  INVESTIGATION 

In  the  event  a  major  mishap  does  occur  within  the  surface  Navy,  an 
investigation  is  required.  The  purpose  of  this  investigation  is  to  “reveal  hazards 
that  could  cause  future  mishaps”  (Department  of  the  Navy,  1-1, 2005). 

In  a  perfect  world,  Reason’s  model  (Figure  5)  simply  works  in  reverse,  in 
that  an  investigation  is  concerned.  A  loss  is  realized  (mishap  itself),  and  then 
analyzed  to  determine  what  latent  and  active  failures  in  the  layers  of  defenses 
occurred  to  identify  the  potential  hazard  (Reason,  1997). 

The  unfortunate  reality,  as  Schmorrow  accurately  explained,  is  that 
accident  investigation  does  not  happen  in  a  perfect  world,  or  vacuum.  Far  from  it, 
in  fact.  Accident  investigation  is  influenced  by  many  factors  including  (but  not 
limited  to)  inherent  bias,  time  constraints  and  the  post-hoc  nature  of  the 
investigation  itself,  as  well  as  the  accident-reporting  model  being  used 
(Schmorrow,  1998). 

According  to  Schmorrow,  “the  perceptions  of  individual  accident 
investigators  can  confound  the  goals  of  an  accident  investigation”  (Schmorrow, 
1998,  p.  14).  For  instance,  a  civilian  engineer  looking  at  a  collision  will  almost 
instinctively  focus  on  the  most  familiar  systems.  This  bias  could  lead  to 
overlooking  HSI  issues  that  actually  contributed  to  the  mishap.  Additionally, 
previous  experience  or  inexperience  with  particular  types  of  accidents  can  lead  to 
incorrect  conclusions.  If  experience  tells  the  investigator  that  80-90%  of 
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accidents  have  been  attributed  to  human  error,  a  predisposition  may  find  fault  in 
crew  members  where  it  may  not  exist  (Schmorrow,  1998). 

Time  and  timing  also  play  significant  parts  in  the  investigatory  process.  As 
investigations  are  generally  only  conducted  in  relation  to  catastrophic  events, 
such  as  the  collision  or  grounding  of  a  ship,  the  pressure  to  conduct  the 
investigation  in  a  timely  manner  is  significant.  The  Navy,  as  would  any 
organization,  wants  to  find  out  what  went  wrong  to  prevent  that  loss  from 
happening  again,  which  can  cause  undue  stress  on  the  investigators,  and 
potentially  lead  them  to  overlook  or  miss  something.  Additionally,  the  post-hoc 
nature  of  the  process  itself  can  hamper  the  truth.  If  a  member  or  members  of  the 
bridge  watch  team  were  intoxicated  at  the  time  of  the  incident,  but  not  given  a 
breathalyzer  test  at  the  time,  it  may  not  be  possible  to  prove  that  alcohol 
contributed  to  the  incident.  Additionally,  part  of  the  nature  of  the  Navy  is  the  sea, 
the  very  environment  in  which  it  operates.  Tides  and  currents  can  quickly  and 
easily  wash  away  evidence  that  may  be  vital  to  recreate  the  story  of  what 
happened. 

The  last  major  factor  of  an  accident  investigation  is  the  accident-reporting 
model  being  used.  Various  forms,  models,  formats,  and  procedures  are  prevalent 
in  the  field  of  accident  investigation.  This  raises  at  least  two  key  questions.  First, 
what  if  the  investigation  produces  results  not  consistent  with  the  reporting  model? 
Second,  if  the  model  tells  the  investigators  what  they  are  “supposed”  to  find,  will 
they  then  shape  their  results  to  fit  that  model? 

C.  HFACS 

HFACS  was  developed  by  Shappell  and  Wiegmann  and  is  based  on 
Reason’s  (1990)  previously  described  model  of  human  error.  The  purpose  of 
HFACS  is  to  establish  a  “comprehensive,  user-friendly  tool  for  identifying  and 
classifying  the  human  causes  of  aviation  accidents”  (Shappell  &  Wiegmann, 
2001).  Originally  developed  for  use  in  the  Naval  Service  (Navy  and  Marine  Corps 
aviation),  HFACS  is  now  required  to  be  used  across  all  branches  of  service  for 
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the  classification  of  human  error  in  accidents  (DoD,  2005).  The  original  version 
has  been  adapted  to  an  all-inclusive  version  that  can  be  used  in  land,  air, 
surface,  and  sub-surface  accidents. 

Several  other  methods  of  accident  classification  currently  in  use  in  the 
civilian  sector  are  worth  mentioning.  Accimap  was  developed  by  Rasmussen  in 
1997  and  divides  safety  within  a  given  system  into  levels  consisting  of 
government  policy  and  budgeting,  regulatory  bodies  and  associations,  local  area 
government  planning  and  budgeting,  technical  and  operational  management, 
physical  processes  and  actor  activities,  and  equipment  and  surroundings.  This 
method  of  error  analysis  is  generic  and  does  not  use  a  taxonomy  (Salmon  et  al., 
2012). 

STAMP  is  a  second  method  of  mishap  classification,  and  focuses  on 
control  as  the  primary  reason  for  failures.  These  controls  are  divided  into 
managerial,  organizational,  physical,  operational,  and  manufacturing.  The  final 
description  produced  by  this  method  highlights  the  overall  control  structure  of  a 
system,  and  which  parts  yielded  the  failure  in  question  (Salmon  et  al.,  2012). 

As  HFACS  is  the  only  error  taxonomy  currently  in  use  by  the  DoD,  it  will 
be  the  focus  of  this  research. 

1.  Structure  and  Usage 

HFACS  bridges  the  gap  between  Reason’s  theory  and  the  actual  practice 
of  classifying  human  error  in  accident  investigation.  To  this  end,  the  HFACS 
framework  divides  Reason’s  model  into  four  levels  of  human  error:  organizational 
influences,  supervision,  preconditions,  and  acts.  Listed  under  each  of  these 
categories  are  nano-codes  that  allow  for  greater  specificity  as  to  the  nature  of  the 
latent  or  active  failure  that  contributed  to  the  mishap. 
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a.  Organizational  Influences 

Organizational  Influences  (Figure  5)  fall  under  Reason’s  latent 
failures.  Decisions  made  by  numbered  Fleet  Commanders,  Type  Commanders 
(TYCOMS),  and  even  Immediate  Superiors  in  Command  (ISICs)  can  eventually 
lead  to  mishaps. 


Figure  5.  Organizational  factors  influencing  accidents 


Resource  and  Acquisition  Management  refers  to  decision  making 
regarding  equipment  purchases,  upgrades,  upkeep,  and  general  fiscal 
management.  Examples  of  nano-codes  include  inadequate  personnel  recruiting 
policies,  insufficient  support  facilities  and  equipment,  failure  to  provide  sufficient 
funding,  failure  to  remove  or  upgrade  antiquated  equipment,  and  purchasing 
poorly  designed  or  unsuitable  equipment  (Naval  Safety  Center,  2007). 

Organizational  climate  refers  to  the  “atmosphere”  within  an 
organization.  A  command’s  climate  often  tells  a  great  deal  about  it. 
Organizational  climate  issues  influencing  mishaps  may  include  over-confidence 
in  equipment,  unclear  organizational  structure,  and  undue  pressure  or  demand 
for  mission  accomplishment  (Naval  Safety  Center,  2007). 

Similarly,  the  processes  of  an  organization  may  set  up  commands 
in  the  lower  echelons  for  failure.  Unsafe  conditions  due  to  high  operational 
tempo,  inadequate  procedural  guidance,  unsatisfactory  program  management,  or 
lack  of  formal  training  can  all  have  long-term  and  unintended  impacts  (Naval 
Safety  Center,  2007). 
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b.  Supervision 

Supervision,  shown  in  Figure  6,  more  specifically  at  the  command 
level,  has  a  direct  impact  on  safety  and  risk  management  within  that  command. 
Supervisors  failing  to  adhere  to  rules  and  regulations,  as  well  as  failing  to  require 
their  subordinates  to  do  the  same,  may  set  their  commands  up  for  catastrophic 
failure. 


Figure  6.  Categories  of  unsafe  supervision 


Inadequate  supervision  and  leadership  can  quickly  lead  to  disaster. 
In  high  stress  situations  or  instances  when  subordinates  are  relatively  unfamiliar 
with  the  unfolding  situation  (inexperienced),  supervision  is  instrumental  in 
preventing  potential  mishaps.  Lack  of  training,  guidance,  policy,  and  even 
personality  conflicts  are  examples  of  inadequate  supervision. 

Risk  arises  when  inappropriate  operations  are  planned.  If  a 
supervisor  selects  an  individual  without  the  requisite  experience  level  for  a  task, 
authorizes  an  unnecessary  hazard,  or  directs  actions  to  be  taken  outside  the 
capabilities  of  equipment,  a  mishap  is  likely  to  follow. 

Similarly,  it  is  incumbent  upon  supervisors  to  correct  issues  brought 
to  light.  Failing  to  correct  risky  behavior  or  unsafe  practices  by  subordinates  can 
have  catastrophic  consequences. 

Lastly,  violating  or  intentionally  disregarding  guidance  or  policies 
creates  undue  risk  within  a  command.  Failing  to  enforce  rules,  espousing  “tribal 
knowledge”  over  written  instructions,  or  directing  violations  of  standard  policies, 
create  risk  that  can  lead  to  eventual  disaster  (Naval  Safety  Center,  2007). 
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c. 


Preconditions 


Latent  or  potential  hazards  exist  all  around.  As  Figure  7  helps 
illustrate,  in  a  high-tempo  and  complex  organization,  such  as  the  surface  Navy, 
both  the  physical  and  technical  environment  can  play  significant  roles  in  causing 
mishaps.  Personal  issues  existing  within  individuals  and  among  individuals  in  an 
organization  also  can  contribute  to  mishaps. 
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Figure  7.  Categories  of  preconditions  for  unsafe  acts 
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(1)  Environmental  Factors.  On  the  environmental  side, 
weather  and  the  ambient  environment  within  a  ship  are  rife  with  latent  hazards.  In 
addition,  flaws  in  equipment  design  can  lie  dormant  for  months  or  years  but  can 
eventually  contribute  to  a  mishap.  Cold  and  heat  stress,  restricted  visibility, 
lighting  or  backscatter,  equipment  interface  issues  (HFE),  and  instrumentation 
and  warning  issues  are  possible  technical  preconditions  for  a  mishap. 

(2)  Condition  of  Individuals.  The  largest  set  of 
preconditions  comes,  unsurprisingly,  from  individuals.  Such  factors  include 
physical  or  mental  limitations,  cognitive  factors,  adverse  physical  states,  psycho- 
behavioral  factors,  and  perceptual  factors. 

Physically,  humans  have  limitations.  Be  it  with  memory, 
learning  rate,  coordination,  or  even  body  size,  an  individual’s  capabilities,  or  lack 
thereof,  can  be  a  precursor  for  failure,  given  the  right  situation. 

Issues  with  how  an  individual  perceives  a  given  situation  can 
prove  to  be  risky  as  well.  Spatial  disorientation,  coupled  with  misinterpreting  or 
misreading  instruments,  and  misperceiving  a  changing  environment  can  cause 
individuals  to  respond  incorrectly  for  a  given  situation,  eventually  leading  to 
disaster. 

Even  an  individual’s  personality,  motivation  level,  and  other 
psychosocial  issues  or  psychological  disorders  can  prove  to  be  a  source  of  risk 
given  the  proper  situation.  Emotional  state,  excess  aggression,  overconfidence, 
and  complacency  are  potential  factors  within  individuals  that  can  impact 
decisions  and  create  added  risk. 

The  final  precondition  within  individuals  deals  directly  with 
physiological  states.  Existing  medical  or  physiological  conditions  include  the 
effects  of  prescribed  drugs,  overexertion,  motion  sickness,  and  dehydration. 
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(3)  Personnel  Factors.  The  last  category  of  preconditions 
exists  among  personnel.  Communication,  coordinating  and  planning,  as  well  as 
self-imposed  stress,  must  be  considered  factors  during  accident  investigation,  as 
they  can  play  a  major  role  in  mishaps.  Self-imposed  stress  takes  many  forms. 
Alcohol,  improper  diet,  illegal  drugs,  and  even  the  fitness  level  of  personnel  can 
all  become  precursors  for  serious  incidents,  given  the  right  prevailing 
circumstances. 

One  of  the  largest  and  most  common  sections  of  precursors 
is  those  of  coordination,  communication  and  planning.  Lack  of  assertiveness, 
failure  to  communicate  key  information,  inadequate  planning,  as  well  as  failing  to 
re-assess  situations  as  they  begin  to  change,  can  all  lead  to  mishaps  (Naval 
Safety  Center,  2007). 

d.  Acts 

Acts  are  shown  in  Figure  8.  Acts  are  the  actions  or  decisions  that 
directly  lead  to  an  accident.  Acts,  or  unsafe  acts,  are  categorized  within  DoD 
HFACS  as  either  errors  or  violations. 


Figure  8.  Categories  of  unsafe  acts 


2.  Errors 

Errors  come  in  three  forms:  Skill-based,  judgment  and  decision  making, 
and  perception.  According  to  Shappell  and  Wiegmann,  “errors  represent  the 
mental  or  physical  activities  of  individuals  that  fail  to  achieve  their  intended 
outcome”  (Shappell  &  Wiegmann,  2001,  p.  62).  Skill-based  errors  are  generally 


22 


fairly  routine  standard  activities  conducted  by  individuals.  Such  errors  in  skill 
include  over-control,  not  following  a  checklist  or  procedure,  and  unintended 
operation  of  specific  equipment.  Errors  in  judgment  and  decision  making  involve 
choosing  the  wrong  course  of  action  in  a  given  situation.  These  errors  can  take 
the  form  of  rushing  necessary  actions,  delaying  a  necessary  action,  or  ignoring 
cautions  or  warning.  Whatever  form  they  take,  these  errors  can  nevertheless 
prove  costly.  Perception  errors,  or  errors  due  to  misperception,  occur  due  to  an 
incorrect  response  to  an  individual’s  senses. 

Violations  are  the  second  form  of  unsafe  acts  and  occur  when  rules  are 
broken  or  instructions  are  not  followed.  Violations  occur  in  the  form  of  work¬ 
around  violations,  extreme  or  willful  violations  by  an  individual,  or  even 
widespread  and  routine  violations  (Naval  Safety  Center,  2007). 

3.  HFACS  Application  and  Research 

Since  its  creation,  the  HFACS  taxonomy  has  been  widely  used,  modified, 
and  scrutinized.  Due  to  the  high  cost  of  mishaps  within  industrial  and  commercial 
sectors,  it  makes  sense  for  organizations  to  seek  out  a  system,  such  as  HFACS, 
to  classify  and  count  errors  better  for  more  effective  prevention.  However,  one 
size  does  not  often  fit  all.  To  this  end,  many  researchers  have  adapted  HFACS 
from  its  original  form  for  use  in  areas,  such  as  shipboard  machinery  spaces,  the 
mining  industry  in  Australia,  and  even  a  version  for  use  with  railroad  error 
investigation.  However,  a  fair  number  of  skeptics  remain  who  doubt  the  rating 
consistency  (i.e.,  inter-rater  reliability)  when  HFACS  is  used  in  mishap 
investigations. 

More  than  80  articles  have  been  published  on  HFACS  since  its  inception. 
The  preponderance  of  this  research  presumes  HFACS  to  be  a  valid,  verified 
taxonomy,  and  use  it  as  such.  At  the  time  of  their  paper  in  2001,  Shappell  and 
Wiegmann  cited  inter-rater  reliabilities  from  five  studies  with  an  average 
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consensus  of  between  .6  and  .95  for  a  variety  of  studies.  It  should  be  noted  that 
these  studies  were  only  coded  at  the  categorical  level,  and  not  at  a  nano-code 
level  (Shappell  &  Wiegmann,  2001). 

With  such  claims  of  reliability,  it  is  not  surprising  that  many  researchers 
accept  HFACS  at  face  value.  Lenne  and  his  colleagues’  work  with  safety  in  the 
Australian  mining  community  used  the  original  17  categories  to  code  263  mining 
incidents  in  Australia  from  2007  to  2008  (Lenne,  Salmon,  Liu,  &  Trotter,  2011). 
This  study  used  pairs  of  human  factors  researchers  to  translate  codes  from  an 
Incident  Cause  Analysis  Method  (IDCAM)  model  into  HFACS  codes.  Although 
the  researchers  coded  independently,  they  resolved  decision  differences  in  a 
consensus  method,  much  as  a  SIB  would  do  (Lenne  et  al.,  2011).  Studies  using 
methods  such  as  these  avoid  the  labor  of  calculating  inter-rater  reliability  by 
ceding  validity  to  the  HFACS  model. 

In  2010,  Wertheim  used  HFACS  to  look  at  human  error  in  large-scale 
biometric  systems.  In  this  research,  the  use  of  HFACS  was  shown  to  improve 
fingerprint  match  rate  by  as  much  as  10%.  Again,  however,  HFACS  was 
assumed  to  be  valid  and  no  inter-rater  reliability  was  not  questioned  (Wertheim, 
2010). 

Like  Lenne  et  al.  (2007),  the  Australian  Transport  Safety  Bureau  chose  a 
similar  path  when  examining  accidents  within  the  Australian  civil  aviation 
community  in  2007.  This  study  again  coded  accidents  using  pairs  of  raters.  This 
consensus  method  is  the  most  common  method  currently  in  use  among  users  of 
HFACS  according  to  the  research  available  (Inglis,  Sutton,  &  McRandle,  2007). 

Over  the  years,  new  versions  of  HFACS  have  been  developed.  As  the 
desire  of  organizations  to  narrow  down  and  eliminate  causal  factors  of  accidents 
has  increased,  so  has  the  specificity  of  HFACS.  The  preponderance  of  HFACS 
variants  focus  on  developing  systems  at  the  categorical  level,  with  only  one 
version  daring  to  venture  into  the  nano-code  level.  Schroder-Hinrichs  and  his 
colleagues  developed  a  version  of  HFACS  for  machinery  spaces  on  commercial 
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vessels.  HFACS-MSS,  as  it  is  called,  attempts  to  add  specificity  to  the  domain  of 
machinery  accidents  on  sea-going  vessels.  HFACS-MSS  adds  an  additional 
category,  outside  factors,  and  changes  some  of  the  third  tier  categories  from  their 
original  form  in  HFACS  to  increase  specificity  (Schroder-Hinrichs,  Baldauf,  & 
Ghirxi,  2011). 

Similarly,  Patterson  and  Shappell  developed  HFACS-MI  for  the  Australian 
mining  industry.  This  version,  like  that  of  Schroder-Hinrichs’s,  adds  a  fifth 
category  of  outside  factors,  which  includes  regulatory  factors  that  may  influence 
future  mishaps  (Patterson  &  Shappell,  2010). 

In  2007,  Reinach  et  al.  (2007)  took  HFACS  research  a  step  further  by  both 
developing  HFACS-RR  for  the  Federal  Railroad  Administration  and  creating  a 
software  tool  to  perform  much  of  the  work.  The  Human  Error  Investigation 
Software  Tool  was  created  to  effectively  “do”  HFACS.  The  program  includes 
checklists,  guides,  a  taxonomy,  and  definitions  to  assist  raters  in  the  process  of 
error  investigation  and  identification  (Reinach,  Viale,  &  Green,  2007). 

Despite  the  bulk  of  HFACS  research  being  generally  positive,  skeptics 
remain.  It  only  makes  sense  that  when  working  for  organizations  prepared  to 
spend  millions  of  dollars  to  reduce  risk  and  mishaps  that  studies  should  focus  on 
validating  the  method  of  risk  identification  actually  being  used. 

O’Connor  is  perhaps  the  most  well  published  skeptic  of  HFACS  from  an 
inter-rater  perspective.  He  has  published  three  papers  examining  the  reliability  of 
HFACS  using  trained  raters  and  simulating  mishap  boards.  O’Connor’s  findings 
demonstrate  general  unreliability  in  the  usability  of  HFACS  for  several  reasons. 
O’Connor  cites  training,  experience,  and  format  as  possible  issues  with  DoD 
HFACS  (O’Connor,  2008;  O’Connor,  Walliser,  &  Philips,  2010;  O’Connor  & 
Walker,  2011). 
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In  a  study  published  in  2011,  Wang  et  al.  put  HFACS  to  the  test  using  air 
traffic  controllers  and  human  factors  experts.  Using  19  HFACS  categories,  the 
study  showed  agreement  percentages  below  40%  for  both  groups  just  at  the 
categorical  level.  No  testing  of  nano-codes  was  conducted  (Wang  et  al.,  2011). 

Lastly,  in  one  of  the  few  studies  to  attempt  an  adaptation  or  revision  of 
HFACS  at  the  nano-code  level,  Olsen  and  Shorrock  found  results  similar  to  that 
of  Wang  et  al.  Their  research  showed  inter-rater  reliability  at  the  categorical  level 
to  be  under  50%  (Olsen  &  Shorrock,  2010). 

DoD  HFACS  is  used  throughout  the  U.S.  military,  as  well  as  organizations 
around  the  world.  It  is  not,  however,  a  perfect  system.  Research  continues  to 
highlight  the  positive  nature  of  HFACS,  but  also  the  negative  issues  associated 
with  its  use. 

The  largest  strength  of  HFACS  lies  in  its  wide  applicability  and  ability  to  be 
adapted  to  other  uses.  One  of  the  best  ways  to  determine  the  relative  usefulness 
of  any  method  is  to  test  it  against  others  that  claim  to  accomplish  a  similar  task. 
Salmon’s  research  in  2012  compared  HFACS  with  STAMP  and  Accimap,  two 
other  systems  for  error  analysis.  According  to  the  study,  HFACS  “lends  itself  to 
multiple  accident  case  analyses,  and  so  is  perhaps  more  suited  to  inclusion  in 
safety  management  systems”  (Salmon  et  al.,  2012). 

Based  on  the  literature  review,  the  largest  strength  of  HFACS  is  perhaps 
also  the  greatest  weakness  of  HFACS.  As  the  system  is  rather  generic,  it  lacks 
domain  specificity,  as  pointed  out  by  Salmon  et  al.  and  Griggs  (Salmon  et  al., 
2012,  Griggs,  2012). 

Additionally,  while  the  system  is  adaptable  and  able  to  be  transformed 
based  on  the  requirements  of  the  domain,  such  a  process  is  difficult  if  the  system 
has  already  been  in  place.  Transforming  the  resulting  codes  from  hundreds, 
perhaps  thousands,  of  incidents  for  input  into  a  database  would  require  many 
man-years  to  re-read  incident  reports  and  re-classify  each  finding. 
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D.  THE  NEED  FOR  HFACS  MARITIME  (HFACS-M) 

The  generic  nature  of  DoD  HFACS  as  a  one-size-fits-all  model  is 
insufficient  for  military  components,  nearly  all  of  which  have  domain-specific 
factors  associated  with  them.  To  improve  reliability,  the  specificity  of  DoD  HFACS 
must  improve  with  regard  to  the  surface  Navy.  To  this  end,  a  maritime  version  of 
HFACS,  HFACS-M,  was  developed.  This  version  will  greatly  serve  the  fleet  by 
more  accurately  and  efficiently  identifying  human  error  components  in  accident 
investigation.  Additionally,  a  more  fleet-centric  version  of  HFACS  will  improve 
usability  of  HFACS  and  make  it  more  suited  for  lower  category  mishaps.  Finally, 
domain-specific  terminology  will  reduce  the  training  time  required  for  novices  to 
become  familiar  with  HFACS. 

The  next  chapter  describes  the  development  of  HFACS-M  and  the  method 
used  to  test  DoD  HFACS  and  HFACS-M. 
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III.  METHOD 


A.  RESEARCH  APPROACH 

This  study  sought  to  compare  the  inter-rater  reliability  among  trained 
raters  when  using  either  HFACS  or  HFACS-M  error  classification  taxonomy  to 
code  a  mishap  report.  Subjects  each  received  standardized  training  via  a  self- 
paced,  pre-recorded,  voice-over  presentation,  which  provided  familiarization  with 
the  respective  taxonomy.  Each  subject  next  read  through  an  executive  summary 
of  a  report  from  the  National  Transportation  Safety  Board  (NTSB).  Subjects  were 
asked  to  review  the  1 1  findings  associated  with  the  mishap,  and  assign 
appropriate  codes  to  each  finding  based  on  their  understanding  of  the  respective 
taxonomy.  Analysis  was  then  conducted  to  determine  the  inter-rater  reliability 
within  each  of  the  two  taxonomies,  as  well  as  the  inter-rater  reliability  between 
SWOs  and  non-SWOs. 

B.  PARTICIPANTS 

A  total  of  28  Naval  Postgraduate  School  students,  all  U.S.  military  officers 
participated  in  this  study.  Gender  and  age  were  not  determined  to  be  a  factor  in 
the  error  classification  process  and  were  not  recorded.  Since  DoD  HFACS  is 
intended  for  use  by  all  branches  of  service,  no  service  was  excluded  from 
participating  in  the  study.  Participants  included  members  of  the  Army,  Navy,  Air 
Force,  Marine  Corps,  and  Coast  Guard.  Of  these  participants,  five  who  took  the 
case  study  using  DoD  HFACS  had  participated  in  accident  investigations  (two 
SWOs  and  three  non-SWOs),  and  four  participants  (two  SWOs  and  two  non- 
SWOs)  using  HFACS-M  (described  in  section  C.3)  had  also  participated  in  an 
accident  investigation  at  some  point  in  their  careers.  None  who  claimed  to  have 
participated  in  an  accident  investigation  had  any  experience  with  HFACS  in  the 
course  of  those  investigations.  See  Table  1 . 
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Table  1 .  Two-by-two  experiment  matrix  of  participants  by  HFACS  version 


DOD  HFACS 

HFACS-M 

swo 

7 

7 

NON-SWO 

7 

7 

C.  APPARATUS 

This  study  consisted  of  three  major  pieces:  self-paced  training,  a  case 
study,  and  the  DoD  HFACS  and  HFACS-M  coding  sheets. 

1.  Training 

The  training  was  conducted  via  a  SAKAI  site  and  featured  a  series  of 
PowerPoint  slides  with  associated  voice  recording.  The  presentation  offered  a 
brief  history  of  either  DoD  HFACS  or  HFACS-M,  as  well  as  a  description  of  the 
four  categories  of  each  of  the  taxonomies.  The  latter  portion  of  the  presentation 
featured  a  practice  case  study  with  four  findings  from  a  fictitious  mishap.  The 
training  divided  each  of  the  four  findings  into  its  respective  category  based  on  the 
taxonomy  being  employed.  Subjects  were  required  to  select  the  nano-code  that 
best  described  the  issue  stated  in  the  finding.  The  PowerPoint  slides  can  be 
found  in  Appendix  A.  Figure  9  provides  the  reader  with  an  example  of  one 
PowerPoint  slide  and  its  narration  from  the  DoD  HFACS  training. 
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Organizational  Influences 

•  Below  are  the  nano-codes  associated  with  Organizational  Influences. 

Take  the  time  now  and  select  the  one  which  best  describes  the 
following: 

-  The  parent  company  of  vessel  A  encouraged  a  schedule  that  maximized  profits 
and  did  not  allow  for  proper  crew  rest 
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Below  are  the  nano-codes  associated  with  Organizational  Influences.  Take  the 
time  now  and  select  the  one  which  best  describes  the  following 

The  parent  company  of  vessel  A  encouraged  a  schedule  that  did  not 
allow  for  proper  crew  rest 

Figure  9.  Training  slide  example  with  speaker  notes 

2.  Case  Study 

The  second  portion  of  the  apparatus  was  the  case  study,  which  consisted 
of  the  executive  summary  of  an  actual  mishap  along  with  the  findings  from  the 
mishap.  The  mishap  was  selected  from  the  NTSB  database  based  on  its 
moderate  number  of  findings  and  moderate  level  of  complication.  As  the  NTSB 
has  consistent  mishap  investigation  practices,  it  was  determined  that  in  the 
interest  of  time,  it  would  be  well  suited  for  this  study.  The  accident  report  used  in 
this  study  was  NTSB/MAR-1 1/04,  Collision  of  Tankship  Eagle  Otome  with  Cargo 
Vessel  Gull  Arrow  and  Subsequent  Collision  with  the  Dixie  Vengeance  Tow.  This 
incident  occurred  in  the  Sabine-Neches  Canal,  Port  Arthur,  Texas,  on  January 
23,  2010.  The  executive  summary  reads  as  follows. 
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On  Saturday,  January  23,  2010,  about  0935  central  standard  time, 
the  810-foot-long  oil  tankship  Eagle  Otome  collided  with  the  597- 
foot-long  general  cargo  vessel  Gull  Arrow  at  the  Port  of  Port  Arthur, 
Texas.  A  297-foot-long  barge,  the  Kirby  30406,  which  was  being 
pushed  by  the  towboat  Dixie  Vengeance,  subsequently  collided 
with  the  Eagle  Otome.  The  tankship  was  inbound  in  the  Sabine- 
Neches  Canal  with  a  load  of  crude  oil  en  route  to  an  ExxonMobil 
facility  in  Beaumont,  Texas.  Two  pilots  were  on  board,  as  called  for 
by  local  waterway  protocol.  When  the  Eagle  Otome  approached  the 
Port  of  Port  Arthur,  it  experienced  several  unintended  heading 
diversions  culminating  in  the  Eagle  Otome  striking  the  Gull  Arrow, 
which  was  berthed  at  the  port  unloading  cargo. 

A  short  distance  upriver  from  the  collision  site,  the  Dixie  Vengeance 
was  outbound  with  two  barges.  The  towboat  master  saw  the  Eagle 
Otome  move  toward  his  side  of  the  canal,  and  he  put  his  engines 
full  astern  but  could  not  avoid  the  subsequent  collision.  The  Kirby 
30406,  which  was  the  forward  barge  pushed  by  the  Dixie 
Vengeance,  collided  with  the  Eagle  Otome  and  breached  the 
tankship’s  starboard  ballast  tank  and  the  No.  1  center  cargo  tank  a 
few  feet  above  the  waterline.  As  a  result  of  the  breach,  862,344 
gallons  of  oil  were  released  from  the  cargo  tank,  and  an  estimated 
462,000  gallons  of  that  amount  spilled  into  the  water.  The  three 
vessels  remained  together  in  the  center  of  the  canal  while  pollution 
response  procedures  were  initiated.  No  crewmember  on  board  any 
of  the  three  vessels  was  injured. 

The  National  Transportation  Safety  Board  (NTSB)  determines  that 
the  probable  cause  of  the  collision  of  tankship  Eagle  Otome  with 
cargo  vessel  Gull  Arrow  and  the  subsequent  collision  with  the  Dixie 
Vengeance  tow  was  the  failure  of  the  first  pilot,  who  had 
navigational  control  of  the  Eagle  Otome,  to  correct  the  sheering 
motions  that  began  as  a  result  of  the  late  initiation  of  a  turn  at  a 
mild  bend  in  the  waterway.  Contributing  to  the  accident  was  the  first 
pilot’s  fatigue,  caused  by  his  untreated  obstructive  sleep  apnea  and 
his  work  schedule,  which  did  not  permit  adequate  sleep;  his 
distraction  from  conducting  a  radio  call,  which  the  second  pilot 
should  have  conducted  in  accordance  with  guidelines;  and  the  lack 
of  effective  bridge  resource  management  by  both  pilots.  Also 
contributing  was  the  lack  of  oversight  by  the  Jefferson  and  Orange 
County  Board  of  Pilot  Commissioners. 
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Following  the  executive  summary  was  a  partial  list  of  findings  from  the 
accident  investigation  presented  to  the  participants.  They  read  as  follows. 

Based  on  your  knowledge  of  the  associated  error  classification 
taxonomy  and  your  understanding  of  the  facts  surrounding  the 
investigation,  assign  an  appropriate  nano-code  that  best  describes 
each  of  the  findings  listed  below.  Please  note  that  there  is  no  right 
or  wrong  answer.  Carefully  read  and  consider  the  possible  options 
before  answering. 

1.  The  Eagle  Otome  pilots  did  not  follow  Sabine  Pilots 
Association  guidelines  with  respect  to  division  of  duties  while 
under  way. 

2.  Although  both  pilots  completed  bridge  resource 
management  training,  they  failed  to  apply  the  team 
performance  aspects  of  bridge  resource  management  to  this 
operation. 

3.  Contrary  to  pilot  association  guidelines,  the  first  pilot  on  the 
Eagle  Otome  was  conducting  a  radio  call  at  a  critical  point  in 
the  waterway,  and  the  radio  call  interfered  with  his  ability  to 
fully  focus  on  conning  the  vessel. 

4.  Had  the  Eagle  Otome  pilots  alerted  the  Dixie  Vengeance 
master  of  the  sheering  problem,  the  force  of  the  collision 
between  the  Eagle  Otome  and  the  Dixie  Vengeance  tow 
would  have  been  lessened  or  the  collision  might  have  been 
avoided  altogether. 

5.  The  combination  of  untreated  obstructive  sleep  apnea, 
disruption  to  his  circadian  rhythms,  and  extended  periods  of 
wakefulness  that  resulted  from  his  work  schedule  caused  the 
first  pilot  to  be  fatigued  at  the  time  of  the  accident. 

6.  The  first  pilot’s  failure  to  correct  the  sheering  motions  that 
began  after  his  late  turn  initiation  at  Missouri  Bend  led  to  the 
accident. 

7.  The  first  pilot’s  fatigue  adversely  affected  his  ability  to  predict 
and  stop  the  Eagle  Otome’s  sheering. 

8.  No  effective  hours  of  service  rules  were  in  place  that  would 
have  prevented  the  Sabine  pilots  from  being  fatigued  by  the 
schedules  that  they  maintained. 
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9.  The  absence  of  an  effective  fatigue  mitigation  and 
prevention  program  among  the  pilots  operating  under  the 
authority  of  the  Jefferson  and  Orange  County  Board  of  Pilot 
Commissioners  created  a  threat  to  the  safety  of  the 
waterway,  its  users,  and  those  nearby. 

10.  The  Jefferson  and  Orange  County  Board  of  Pilot 
Commissioners  should  have  more  fully  exercised  its 
authority  over  pilot  operations  on  the  Sabine-Neches 
Waterway  by  becoming  aware  of  and  enforcing  the  Sabine 
Pilots  Association’s  two-pilot  guidelines  and  implementing  a 
fatigue  mitigation  and  prevention  program  among  the  Sabine 
pilots. 

1 1 .  Commonly  accepted  human  factors  principles  were  not 
applied  to  the  design  of  the  Eagle  Otome’s  engine  control 
console,  which  increased  the  likelihood  of  error  in  the  use  of 
the  controls. 

The  following  findings  from  the  mishap  investigation  were  not  presented  to  the 
participants  because  either  they  did  not  actually  address  an  error  or  they 
speculated  on  or  made  recommendations  for  future  improvements. 

•  Weather,  mechanical  failure,  and  illegal  drug  or  alcohol  use  were 
not  factors  in  the  accident. 

•  The  vessel  meeting  arrangement  agreed  to  by  the  towboat  master 
and  the  first  pilot  was  appropriate  and  was  not  a  factor  in  the 
accident. 

•  Personnel  at  Vessel  Traffic  Service  Port  Arthur  played  no  role  in  the 
accident. 

•  The  Coast  Guard  is  the  organization  with  the  resources, 
capabilities,  and  expertise  best  suited  to  (1)  enhance 
communication  among  pilot  oversight  organizations  and 
(2)  establish  an  easy-to-use  and  readily  available  database  of  pilot 
incidents  and  accidents. 

•  The  first  pilot’s  sounding  the  Eagle  Otome’s  whistle  and  the  Gull 
Arrow  master’s  sounding  the  cargo  vessel’s  general  alarm  were 
prudent  and  effective. 

•  The  accident  response  and  oil  spill  recovery  efforts  were  timely  and 
effective. 
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•  The  dimensions  of  the  Sabine-Neches  Waterway  may  pose  an 
unacceptable  risk,  given  the  size  and  number  of  vessels  transiting 
the  waterway. 

•  Consistent  use  of  a  vessel’s  name  in  radio  communication  can  help 
avoid  confusion  and  enhance  bridge  team  coordination 

3.  DoD  HFACS  and  HFACS-M 

Participants  received  training  on  either  DoD  HFACS  or  HFACS-M,  and 
received  corresponding  coding  sheets.  The  categories,  sub-codes,  and  nano¬ 
codes  used  in  the  DoD  HFACS  coding  sheets  were  taken  directly  from  the  Naval 
Safety  Center’s  2007  booklet,  “DoD  Human  Factors  Analysis  and  Classification 
System  (HFACS).” 

The  coding  sheet  was  divided  by  category,  sub-code,  and  nano-code  as 
shown  in  Figure  10.  Each  nano-code  was  given  its  own  row  of  11  boxes 
representing  the  1 1  findings  of  the  accident  investigation. 
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DOD  HFACS  Findings 


Naval  Safety  Center,  2007  version _  Mark  an  X  in  the  box  below  associated  with  your  choice  for  the  best 


ACTS 

fit  nanocode  for  each  of  the  findings 

Nano-Code 

Description 

Skill-Based  Errors 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

AE  101 

Unintended  operation  of  equipment 

AE  102 

Checklist  not  followed  correctly 

AE  103 

Procedure  not  followed  correctly 

AE  104 

Over-Control  led/under-control  led  aircraft/vehicle 

AE  105 

Breakdown  in  visual  scan 

AE  106 

Inadequate  Anti-G  straining  maneuver 

Judgement  and  Decision-Making  Errors 

AE  201 

Inadequate  real-time  risk  assessment  (e.g.,  failure  of  time-critical  ORM) 

AE  202 

Failure  to  prioritize  tasks  adequately 

AE  203 

Rushed  a  necessary  action 

AE  204 

Delayed  a  necessary  action 

AE  205 

Ignored  a  caution/warning 

AE  206 

Wrong  choice  of  action  during  an  operation  (e.g.,  response  to  an  emergency) 

Perception  Errors 

AE  301 

Incorrect  response  to  a  misperception  (e.g.,  visual  illusion  or  spatial  disorientation) 

Violations 

AV  001 

Work-around  violation  (e.g.,  breaking  the  rules  is  prceived  as  the  best  solution) 

AV  002 

Widespread/routine  violation  (e.g.,  habitual  deviation  from  the  rules  that  is  tolerated  by  management) 

AV  003 

Extreme  violation  (e.g.,  a  violation  not  condoned  by  management 

Figure  10.  DoD  HFACS  coding  sheet  example 
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HFACS-M  was  presented  in  the  same  manner  as  DoD  HFACS.  FIFACS-M 
was  created  by  modifying  the  original  2007  version  of  DoD  HFACS  to  make  it 
more  specific  to  the  surface  Navy.  To  this  end,  the  following  modifications  were 
made. 


•  AE1 02 — Rephrased — Checklist  not  followed/not  followed  correctly 

•  AE  103 — Rephrased — Procedure  not  followed/not  followed 
correctly 

•  AE  104 — Rephrased — Over-Controlled  or  under-controlled  vessel 

•  AE  106 — Removed  (N/A  for  shipboard  use) — Inadequate  Anti-G 
straining  maneuver 

•  PE  101 — Rephrased — Icing/fog  on  window  restricts  visibility 

•  PE  1 02 — Rephrased — Weather  conditions  restrict  visibility 

•  PE  103 — Rephrased — Vibrations/rolls  affect  vision  or  balance 

•  PEI 09 — Rephrased — Backlighting/backscatter  interfere  with 
performance 

•  PE1 1 2 — Added — High  winds/Heavy  seas  affect/impair  movement 

•  PE201 — Removed  (N/A  for  shipboard  use) — Seat  and  restraint 
systems  problems 

•  PE208 — Added — Equipment  not  configured  correctly 

•  PE209 — Added — Corrective  maintenance  not  conducted/not 
conducted  correctly 

•  PE210 — Added — Preventive  maintenance  not  conducted/not 
conducted  correctly 

•  PP101 — Rephrased — Failure  of  watchteam/crew  leadership 

•  PC301 — Removed  (N/A  for  shipboard  use) — Effects  of  G  forces 
(e.g.,  G-LOC) 

•  PC  304 — Removed  N/A  parenthesis — Sudden 
incapacitation/unconsciousness  (not  due  to  G) 

•  PC  308 — Rephrased — Circadian  rhythm  de-synchronization  (watch 
rotation  or  shift  work) 

•  PC  310 — Removed  (N/A  for  shipboard  use) — Trapped  gas 
disorders 

•  PC31 1 — Removed  (N/A  for  shipboard  use) — Evolved  gas  disorders 
(e.g.,  decompression  sickness/bends 
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Supervision — Renamed — Command 

•  SI007 — Added — Failed  to  communicate  intent  (e.g.,  standing 
orders/night  orders) 

Manning/Personnel/Training  Issues — Added  new  subcategory 

•  SP007 — Added — Directed  mission  without  sufficient  manning 

•  SP007 — Command  (formal)  training  is  inadequate 

•  SP008 — Rephrased — Performed  inadequate  risk  assessment 
(ORM) 

•  SV004 — Moved  to  MPT  sub-category 

•  OROOI — Rephrased — Port  facilities  are  deficient 

•  OR002 — Channel  markers/lighting  are  deficient 

•  OR005 — Added — Failure  to  procure  new  systems/upgrades  in  a 
timely  manner 

•  OP007 — Organizational  process  provides  inadequate,  untimely 
guidance 

These  changes  were  necessary  to  remove  ambiguity  and  to  fill  gaps  in  DoD 
HFACS  because  of  the  generic  nature  of  the  taxonomy. 

The  coding  sheet  for  HFACS-M  was  also  divided  by  category,  sub-code, 
and  nano-code  as  shown  in  Figure  1 1 .  Again,  each  nano-code  was  given  its  own 
row  of  1 1  boxes  representing  the  1 1  findings  of  the  accident  investigation. 
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HFACS-M  Findings 


Naval  Postgraduate  School,  2013  version _  Mark  an  X  in  the  box  below  associated  with  your  choice  for  the  best 


ACTS 

fit  nanocode  for  each  of  the  findings 

Nano-Code 

Descri  ption 

Skill-Based  Errors 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

AE  101 

Unintended  operation  of  equipment 

AE  102 

Checklist  not  followed/not  followed  correctly 

AE  103 

Procedure  not  followed/not  followed  correctly 

AE  104 

Over-Controlled  or  under-controlled  vessel 

AE  105 

Breakdown  in  visual  scan 

Judgement  and  Decision-Making  Errors 

AE  201 

Inadequate  real-time  risk  assessment  (e.g.,  failure  of  time-critical  ORM) 

AE  202 

Failure  to  prioritize  tasks  adequately 

AE  203 

Rushed  a  necessary  action 

AE  204 

Delayed  a  necessary  action 

AE  205 

Ignored  a  caution/warning 

AE  206 

Wrong  choice  of  action  during  an  operation  (e.g.,  response  to  an  emergency) 

Perception  Errors 

AE  301 

Incorrect  response  to  a  misperception  (e.g.,  visual  illusion  or  spatial  disorientation) 

Violations 

AV  001 

Work-around  violation  (e.g.,  breaking  the  rules  is  prceived  as  the  best  solution) 

AV  002 

Widespread/routine  violation  (e.g.,  habitual  deviation  from  the  rules  that  is  tolerated  by  management) 

AV  003 

Extreme  violation  (e.g.,  a  violation  not  condoned  by  management 

Figure  1 1 .  HFACS-M  coding  sheet  example 
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D.  PROCEDURES 

The  Naval  Postgraduate  School’s  Institutional  Review  Board  (IRB) 
reviewed  and  approved  this  research.  Volunteers  were  recruited  via  email  from 
the  student  body.  They  reported  to  the  Human  Systems  Integration  Laboratory 
and  were  met  by  the  student  researcher.  They  were  asked  to  sit  in  front  of  a 
computer  with  either  the  DoD  HFACS  or  HFACS-M  training  loaded  on  it.  The 
subjects  read  and  signed  the  informed  consent  form  before  proceeding.  Next, 
each  subject  viewed  the  voice-recorded  training  slides.  Subjects  were  instructed 
to  progress  through  the  slides  at  their  own  pace.  Upon  reaching  the  practice 
slides,  subjects  were  instructed  to  read  through  all  the  possible  nano-codes 
before  making  a  selection.  They  were  given  a  pen  and  scratch  paper  with  which 
to  take  notes  as  desired. 

Upon  completion  of  the  training,  each  subject  was  asked  to  answer  the 
following  questions. 

1.  Have  you  completed  the  associated  training?  Yes  No 

2.  Have  you  ever  been  involved  in  an  accident  investigation?  Yes  No 

3.  Have  you  ever  used  HFACS  in  the  course  of  an  accident  investigation?  Yes  No 

4.  What  is  your  current  designator/MOS/AFSC?  _ 

Next,  the  subjects  were  instructed  to  read  the  executive  summary  from  the 
NTSB  accident  report.  Following  this,  they  were  given  the  list  of  1 1  findings  from 
the  accident  report  and  asked  to  assign  one  and  only  one  nano-code  from  the 
taxonomy  they  were  given  that,  in  their  judgment,  best  described  the  finding. 
Once  the  subjects  finished  marking  all  their  selections,  they  were  debriefed  and 
thanked  for  their  assistance. 
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E.  DATA  ANALYSIS 

Upon  completion  of  data  collection,  it  was  determined  that  no  respondent 
data  would  be  excluded.  None  of  the  subjects  had  used  HFACS  previously. 
Although  several  had  been  involved  in  accident  investigations,  it  was  determined 
by  the  research  team  that  the  experience  did  not  give  them  any  significant 
advantage. 

The  tables  completed  by  individual  raters  were  compiled  into  a  data  table. 
A  Fleiss’  Kappa  analysis  was  conducted  to  determine  the  inter-rater  reliability  of 
those  subjects  using  DoD  HFACS  compared  to  those  who  coded  using  HFACS- 
M.  A  Fleiss’  Kappa  analysis  was  also  conducted  to  determine  the  inter-rater 
reliability  between  SWOs  (maritime  domain  experts),  and  non-SWOs.  These 
analyses  were  conducted  at  the  categorical,  sub-code,  and  nano-code  levels. 
Fleiss’  Kappa  was  used  to  determine  inter-rater  reliability  among  multiple  raters, 
rather  than  Cohen’s  Kappa,  which  is  designed  for  only  two  raters  (Fleiss,  1971). 
Following  the  determination  of  Fleiss’  Kappa  for  each  data  set,  a  simulation  was 
conducted  in  R  to  determine  the  significance  of  the  findings.  See  Fleiss  (1971) 
for  a  description  and  explanation  of  Fleiss’  Kappa. 
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IV.  RESULTS 


A.  DESCRIPTION  OF  PARTICIPANTS 

Twenty-eight  Naval  Postgraduate  School  students  took  part  in  this  study. 
Subjects  included  members  from  each  branch  of  service.  Students  self-identified 
their  MOS/AFSC/Designator  in  the  questionnaire  provided.  Table  1  shows  the 
breakdown  of  participants.  All  told,  14  SWOs  and  14  non-SWOS  participated  in 
the  study.  Participants  were  alternated  between  versions  of  HFACS. 

B.  NANO-CODE  ANALYSIS 

Each  participant  selected  one  nano-code  from  either  DoD  HFACS  or 
HFACS-M  for  each  of  the  1 1  findings  in  the  NTSB  investigation.  These  selections 
were  compiled  into  two  tables,  one  for  DoD  HFACS  and  one  for  HFACS-M. 
Participants  1-7  of  Table  2  and  Table  3  were  non-SWOs  and  participants  8-14 
were  SWOs. 
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Table  2.  DoD  HFACS  results  broken  down  by  Designator/MOS/AFSC 


DOD  HFACS 

Findings 

Number 

Desig/MOS/AFSC 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

1 

49/15 

AV001 

PP103 

AE202 

PP106 

PC307 

AE104 

PC307 

OP003 

OP006 

OP002 

PE204 

2 

1310 

PP103 

PP101 

PC108 

PP106 

PC307 

AE204 

PC307 

OPOOl 

OCOOl 

SI001 

PP110 

3 

0602 

AE103 

AE103 

AE206 

PP106 

PC307 

AE206 

PC307 

SI004 

SI004 

SF001 

PE204 

4 

7565 

AE103 

PP101 

AE202 

PP106 

PC307 

AE204 

PC504 

SV002 

SI003 

SF002 

PE204 

5 

49  A 

PP103 

OP004 

PC102 

PP106 

PP205 

PC504 

PC404 

OP002 

OP003 

SI  004 

PE206 

6 

21B/49 

AE202 

AV001 

PC106 

PP106 

SI001 

AE204 

PP205 

OR007 

OP005 

OP006 

OR004 

7 

1810 

PP103 

PP101 

PC106 

PP106 

PC307 

AE103 

AE201 

OCOOl 

SI004 

SI001 

PE204 

8 

1110 

AE103 

OP004 

PC108 

AE204 

PC308 

AE104 

PC308 

SI004 

SF001 

SF002 

PE204 

9 

1110 

SV001 

AE103 

AV001 

AE206 

OPOOl 

AE206 

OPOOl 

OP002 

OP005 

OP002 

OR004 

10 

1110 

SV002 

OP004 

AE103 

PP108 

PC308 

AE206 

PC307 

SI001 

OP002 

OP006 

PE204 

11 

1110 

AE103 

PP103 

PC106 

PP106 

PC308 

AE206 

OCOOl 

OCOOl 

OCOOl 

OP005 

PE204 

12 

1110 

AV001 

AE202 

AE202 

AE204 

PC307 

PP111 

PP205 

SI004 

SF002 

SI001 

PE204 

13 

1110 

AV003 

PP101 

AE206 

PP106 

PC305 

AE104 

PC307 

OP002 

OP003 

OP006 

PE204 

14 

1110 

AE202 

PP102 

SF001 

PP106 

PP206 

AE104 

PC307 

SF001 

OP002 

OP006 

PE207 

Table  3.  HFACS-M  results  broken  down  by  Designator/MOS/AFSC 


HFACS-M 

Findings 

Number 

Desig/MOS/AFSC 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

1 

1317 

AE103 

AE201 

AE201 

PP106 

PC306 

AE104 

PC306 

SI004 

SF002 

SI001 

PE203 

2 

1810 

AE103 

PP112 

PC106 

PP106 

PC304 

AE204 

PC306 

SP201 

OP006 

OP002 

PE203 

3 

0602 

AV002 

SI001 

PC106 

AE204 

PC304 

PC101 

PP205 

OPOOl 

OP002 

OP006 

OR003 

4 

7523 

AE201 

AE103 

PP108 

AE206 

PC307 

AE204 

PC306 

PE204 

PE202 

SI001 

OR004 

5 

1310 

AE103 

PP101 

AE202 

PP106 

PC306 

AE206 

PC306 

OP003 

SI004 

SI006 

OR004 

6 

1120 

AV001 

PP102 

AE203 

PP106 

OPOOl 

AE206 

PC505 

OPOOl 

OR007 

OP006 

PE206 

7 

19A 

AV001 

PP102 

PP101 

PP106 

PC307 

AE204 

PC306 

SF001 

OP006 

SV002 

PE201 

8 

1110 

AV001 

PC206 

PP103 

PP106 

PC304 

AE204 

PC307 

PC  307 

OP002 

OCOOl 

PE201 

9 

1110 

AE103 

AE102 

PC108 

PP106 

PC307 

PP105 

PC307 

OPOOl 

OP002 

OP006 

PE203 

10 

1110 

AE103 

PP101 

PC108 

PP106 

PC306 

AE104 

PC510 

SI004 

SF002 

SI001 

PE208 

11 

1110 

AV001 

PC405 

PC102 

PP106 

PC307 

AE204 

PC306 

OP003 

OP005 

OCOOl 

OR004 

12 

1110 

AE103 

AE206 

AE206 

AE204 

OPOOl 

AE204 

PC306 

OP003 

OCOOl 

OP007 

PE203 

13 

1110 

AE103 

PP101 

PC108 

PP106 

PC306 

AE204 

PC306 

OP002 

OP005 

SI  004 

OP006 

14 

1110 

PC306 

PP101 

PP108 

PP106 

PC306 

AE204 

PC307 

OCOOl 

SP007 

SI001 

OR004 
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1. 


DoD  HFACS 


From  these  results,  tables  were  constructed  to  calculate  Fleiss’  Kappa. 
Table  4  shows  an  example.  Each  nano-code  was  assigned  its  own  column  and 
Pj,  the  proportion  of  assignments  that  were  to  the  y-th  category,  was  calculated 

for  each.  The  rows  delineate  the  finding  with  which  the  code  is  associated.  In 
DoD  HFACS,  147  possible  nano-codes  were  available. 


Table  4.  DoD  HFACS  nano-code  table  example 


DOD NANO 

AE101 

AE102 

AE103 

AE104 

AE105 

AE106 

1 

0 

0 

4 

0 

0 

0 

2 

0 

0 

2 

0 

0 

0 

3 

0 

0 

1 

0 

0 

0 

4 

0 

0 

0 

0 

0 

0 

5 

0 

0 

0 

0 

0 

0 

6 

0 

0 

1 

4 

0 

0 

7 

0 

0 

0 

0 

0 

0 

8 

0 

0 

0 

0 

0 

0 

9 

0 

0 

0 

0 

0 

0 

10 

0 

0 

0 

0 

0 

0 

11 

0 

0 

0 

0 

0 

0 

Total 

0 

0 

8 

4 

0 

0 

Pj 

0.000 

0.000 

0.052 

0.026 

0.000 

0.000 

Fleiss’  Kappa  was  calculated  to  determine  inter-rater  reliability  at  the 
nano-code  level  for  raters  using  DoD  HFACS.  Fleiss’  Kappa  was  derived  by  first 
taking  the  difference  of  P  and  Pe  to  find  the  degree  above  chance  that  was 
achieved.  This  difference  is  then  divided  by  1  -Pe  to  obtain  Fleiss’  Kappa.  The 
overall  results  were  as  follows:  P  =  .185;  Pe  =  .036;  K  =  .154. 

The  results  were  then  divided  between  SWOs  and  non-SWOs.  Fleiss’ 
Kappa  was  calculated  for  each  group  individually.  For  SWOs  employing  DoD 
HFACS  at  the  nano-code  level,  the  results  were  as  follows:  P  =  .147;  Pe  =  .038; 
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K  =  .114.  For  non-SWOs  employing  DoD  HFACS  at  the  nano-code  level,  the 
results  were  as  follows:  P  =  .234;  Pe  =  .045;  K  =  .198. 

2.  HFACS-M 

Fleiss’  Kappa  was  also  calculated  to  determine  inter-rater  reliability  at  the 
nano-code  level  for  raters  using  HFACS-M.  In  HFACS-M,  152  possible  nano¬ 
codes  were  available.  For  HFACS-M  at  the  nano-code  level,  the  overall  results 
were  as  follows:  P  =  .212;  Pe  =  .037;  K  =  .182. 

The  results  were  then  divided  between  SWOs  and  non-SWOs.  Fleiss’ 
Kappa  was  calculated  for  each  group  individually.  For  SWOs  employing  HFACS- 
M  at  the  nano-code  level,  the  results  were  as  follows:  P  =  .238;  Pe=  .046;  K  = 
.202  For  non-SWOs  employing  HFACS-M  at  the  nano-code  level,  the  results 
were  as  follows:  P  =  .169;  Pe  =  .037;  K  =  .137. 

C.  SUB-CODE  LEVEL 

Each  of  the  nano-codes  in  the  DoD  HFACS  and  HFACS-M  taxonomies 
falls  under  a  specific  sub-code.  For  this  next  level  of  analysis,  the  nano-codes 
were  translated  into  their  respective  sub-code  within  the  original  tables. 
Participants  1-7  of  Table  5  and  Table  6  were  non-SWOs  and  participants  8-14 
were  SWOs. 
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Table  5.  DoD  HFACS  sub-codes  broken  down  by  Designator/MOS/AFSC 


DOD  HFACS 

Findings 

Number 

Desig/MOS/AFSC 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

1 

49/15 

A-V 

P-CCPF 

A-JDME 

P-CCPF 

P-APS 

A-SB 

P-APS 

O-P 

O-P 

O-P 

P-TE 

2 

1310 

P-CCPF 

P-CCPF 

P-AF 

P-CCPF 

P-APS 

A-JDME 

P-APS 

O-P 

O-C 

S-IS 

P-CCPF 

3 

0602 

A-SB 

A-SB 

A-JDME 

P-CCPF 

P-APS 

A-JDME 

P-APS 

S-IS 

S-IS 

S-FCKP 

P-TE 

4 

7565 

A-SB 

P-CCPF 

A-JDME 

P-CCPF 

P-APS 

A-JDME 

P-PF 

S-SV 

S-IS 

S-FCKP 

P-TE 

5 

49A 

P-CCPF 

O-P 

P-AF 

P-CCPF 

P-SIS 

P-PF 

P-PML 

O-P 

O-P 

S-IS 

P-TE 

6 

21B/49 

A-JDME 

A-V 

P-AF 

P-CCPF 

S-IS 

A-JDME 

P-SIS 

O-RAM 

O-P 

O-P 

O-RAM 

7 

1810 

A-SB 

P-CCPF 

P-AF 

P-CCPF 

P-APS 

A-SB 

A-JDME 

O-C 

S-IS 

S-IS 

P-TE 

8 

1110 

A-SB 

O-P 

P-AF 

A-JDME 

P-APS 

A-SB 

P-APS 

S-IS 

S-FCKP 

S-FCKP 

P-TE 

9 

1110 

S-SV 

A-SB 

A-V 

A-JDME 

O-P 

A-JDME 

O-P 

O-P 

O-P 

O-P 

O-RAM 

10 

1110 

S-SV 

O-P 

A-SB 

P-CCPF 

P-APS 

A-JDME 

P-APS 

S-IS 

O-P 

O-P 

P-TE 

11 

1110 

A-SB 

P-CCPF 

P-AF 

P-CCPF 

P-APS 

A-JDME 

O-C 

O-C 

O-C 

O-P 

P-TE 

12 

1110 

A-V 

A-JDME 

A-JDME 

A-JDME 

P-APS 

P-CCPF 

P-SIS 

S-IS 

S-FCKP 

S-IS 

P-TE 

13 

1110 

A-V 

P-CCPF 

A-JDME 

P-CCPF 

P-APS 

A-SB 

P-APS 

O-P 

O-P 

O-P 

P-TE 

14 

1110 

A-JDME 

P-CCPF 

S-FCKP 

P-CCPF 

P-SIS 

A-SB 

P-APS 

S-FCKP 

O-P 

O-P 

P-TE 
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Table  6.  HFACS-M  sub-codes  broken  down  by  Designator/MOS/AFSC 


HFACS-M 

Number 

Desig/MOS/AFSC 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

1 

1317 

A-SB 

A-JDME 

A-JDME 

P-CCPF 

P-APS 

A-SB 

P-APS 

C-IS 

C-FCKP 

C-IS 

P-TE 

2 

1810 

A-SB 

P-CCPF 

P-AF 

P-CCPF 

P-APS 

A-JDME 

P-APS 

C-PIO 

O-P 

O-P 

P-TE 

3 

0602 

A-V 

C-IS 

P-AF 

A-JDME 

P-APS 

P-AF 

P-SIS 

O-P 

O-P 

O-P 

O-RAM 

4 

7523 

A-JDME 

A-SB 

P-CCPF 

A-JDME 

P-APS 

A-JDME 

P-APS 

P-TE 

P-TE 

C-IS 

O-RAM 

5 

1310 

A-SB 

P-CCPF 

A-JDME 

P-CCPF 

P-APS 

A-JDME 

P-APS 

O-P 

C-IS 

C-IS 

O-RAM 

6 

1120 

A-V 

P-CCPF 

A-JDME 

P-CCPF 

O-P 

A-JDME 

P-PF 

O-P 

O-RAM 

O-P 

P-TE 

7 

19A 

A-V 

P-CCPF 

P-CCPF 

P-CCPF 

P-APS 

A-JDME 

P-APS 

C-FCKP 

O-P 

C-SV 

P-TE 

8 

1110 

A-V 

P-PBF 

P-CCPF 

P-CCPF 

P-APS 

A-JDME 

P-APS 

O-P 

O-P 

O-C 

P-TE 

9 

1110 

A-SB 

A-SB 

P-AF 

P-CCPF 

P-APS 

P-CCPF 

P-APS 

O-P 

O-P 

O-P 

P-TE 

10 

1110 

A-SB 

P-CCPF 

P-AF 

P-CCPF 

P-APS 

A-SB 

P-PF 

C-IS 

C-FCKP 

C-IS 

P-TE 

11 

1110 

A-V 

P-PML 

P-AF 

P-CCPF 

P-APS 

A-JDME 

P-APS 

O-P 

O-P 

O-C 

O-RAM 

12 

1110 

A-SB 

A-JDME 

A-JDME 

A-JDME 

O-P 

A-JDME 

P-APS 

O-P 

O-C 

O-P 

P-TE 

13 

1110 

A-SB 

P-CCPF 

P-AF 

P-CCPF 

P-APS 

A-JDME 

P-APS 

O-P 

O-P 

C-IS 

O-P 

14 

1110 

P-APS 

P-CCPF 

P-CCPF 

P-CCPF 

P-APS 

A-JDME 

P-APS 

O-C 

C-MPT 

C-IS 

O-RAM 
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1. 


DoD  HFACS 


From  these  results,  tables  were  constructed  to  calculate  Fleiss’  Kappa  at 
the  sub-code  level.  Each  sub-code  was  once  again  assigned  its  own  column  and 
Pj  was  calculated  for  each.  In  DoD  HFACS,  20  possible  sub-codes  were 

available.  Table  7  shows  the  overall  breakdown  of  sub-codes,  shown  in  the 
columns,  and  findings,  represented  by  the  rows. 
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Table  7.  Overall  DoD  HFACS  sub-code  table 


DOD SUB 

A-SB 

A-JDME 

A-PE 

A-V 

P-PE 

P-TE 

P-SIS 

P-CCPF 

P-AF 

P-PML 

P-PF 

P-PBF 

P-APS 

S-iS 

S-FCKP 

S-PIO 

s-sv 

O-RAM 

o-c 

O-P 

Pi 

1 

5 

2 

0 

3 

0 

0 

0 

2 

0 

0 

0 

0 

0 

0 

0 

0 

2 

0 

0 

0 

0.176 

2 

2 

1 

0 

1 

0 

0 

0 

7 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

3 

0.275 

3 

1 

5 

0 

1 

0 

0 

0 

0 

6 

0 

0 

0 

0 

0 

1 

0 

0 

0 

0 

0 

0.275 

4 

0 

3 

0 

0 

0 

0 

0 

11 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0.637 

5 

0 

0 

0 

0 

0 

0 

2 

0 

0 

0 

0 

0 

10 

1 

0 

0 

0 

0 

0 

1 

0.505 

6 

5 

7 

0 

0 

0 

0 

0 

1 

0 

0 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0.341 

7 

0 

1 

0 

0 

0 

0 

2 

0 

0 

1 

1 

0 

7 

0 

0 

0 

0 

0 

1 

1 

0.242 

8 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

4 

1 

0 

1 

1 

2 

5 

0.187 

9 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

3 

2 

0 

0 

0 

2 

7 

0.286 

10 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

4 

3 

0 

0 

0 

0 

7 

0.330 

11 

0 

0 

0 

0 

0 

11 

0 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

2 

0 

0 

0.615 

Total 

13 

19 

0 

5 

0 

11 

4 

22 

6 

1 

2 

0 

17 

12 

0 

3 

3 

3 

24 

3.868 

[Pj_  0.084  0.123  0.000  0.032  0.000  0.071  0.026  0.143  0.039  0.006  0.013  0.000  0.110  0.078  0.045  0.000  0.019  0.019  0.032  0.156 
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Fleiss’  Kappa  was  calculated  to  determine  inter-rater  reliability  at  the  sub¬ 
code  level  for  raters  using  DoD  HFACS.  The  results  were  as  follows:  P  =  .352; 
Pe  =  .098;  K  =  .281. 

Table  7  was  then  divided  between  SWOs  and  non-SWOs.  Fleiss’  Kappa 
was  calculated  for  each  group  individually.  For  SWOs  employing  DoD  HFACS  at 
the  sub-code  level,  the  results  were  as  follows:  P  =  .329;  Pe  =  .106;  K  =  .250 
For  non-SWOs  employing  DoD  HFACS  at  the  sub-code  level,  the  results  were  as 
follows:  P  =  .364;  Pe  =  .099;  K  =  .293. 

2.  HFACS-M 

Fleiss’  Kappa  was  also  calculated  to  determine  inter-rater  reliability  at  the 
sub-code  level  for  raters  using  HFACS-M.  In  HFACS-M,  21  possible  sub-codes 
were  available.  Table  8  shows  the  overall  breakdown  of  sub-codes,  shown  in  the 
columns,  and  findings,  represented  by  the  rows. 


51 


Table  8.  HFACS-M  sub-code  table 


HFACS-M  SUB 

A-SB 

A-JDME 

A-PE 

A-V 

P-PE 

P-TE 

P-SIS  | 

P-CCPF 

P-AF 

P-PML 

P-PF 

P-PBF 

P-APS 

C-IS 

C-FCKP 

C-MPT 

C-PIO 

c-sv 

O-RAM 

O-C 

O-P 

Pi 

1 

7 

1 

0 

5 

0 

0 

0 

0 

0 

0 

0 

0 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0.341 

2 

2 

2 

0 

0 

0 

0 

0 

7 

0 

1 

0 

1 

0 

1 

0 

0 

0 

0 

0 

0 

0 

0.253 

3 

0 

4 

0 

0 

0 

0 

0 

4 

6 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0.297 

4 

0 

3 

0 

0 

0 

0 

0 

11 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0.637 

5 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

12 

0 

0 

0 

0 

0 

0 

0 

2 

0.736 

6 

2 

10 

0 

0 

0 

0 

0 

1 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0.505 

7 

0 

0 

0 

0 

0 

0 

1 

0 

0 

0 

2 

0 

11 

0 

0 

0 

0 

0 

0 

0 

0 

0.615 

8 

0 

0 

0 

0 

0 

1 

0 

0 

0 

0 

0 

0 

0 

2 

1 

0 

1 

0 

0 

1 

8 

0.319 

9 

0 

0 

0 

0 

0 

1 

0 

0 

0 

0 

0 

0 

0 

1 

2 

1 

0 

0 

1 

1 

7 

0.242 

10 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

6 

0 

0 

0 

1 

0 

2 

5 

0.286 

11 

0 

0 

0 

0 

0 

8 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

5 

0 

1 

0.418 

Total 

11 

20 

° 

5 

_ °L 

10 

_ A 

23 

7 

_ iL 

2 

1 

_ 2A. 

10 

3 

1 

1 

1 

6 

23 

4.648 

Pj  0.071  0.130  0.000  0.032  0.000  0.065  0.006  0.149  0.045  0.006  0.013  0.006  0.156  0.065  0.019  0.006  0.006  0.006  0.039  0.026  0.149 
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Fleiss’  Kappa  was  calculated  to  determine  inter-rater  reliability  at  the 
sub-code  level  for  raters  using  HFACS-M.  The  overall  results  were  as  follows: 
P  =  .423;  Pe  =  .105;  K  =  .355. 

Table  8  was  then  divided  between  SWOs  and  non-SWOs.  Fleiss’  Kappa 
was  calculated  for  each  group  individually.  For  SWOs  employing  HFACS-M  at 
the  sub-code  level,  the  results  were  as  follows:  P  =  .433;  Pe  =  .111;  K  =  .362 
For  non-SWOs  employing  HFACS-M  at  the  sub-code  level,  the  results  were  as 
follows:  P  =  .364;  Pe  =  .105;  K  =  .289. 

D.  CATEGORICAL  LEVEL 

Each  of  the  sub-codes  in  the  DoD  HFACS  and  HFACS-M  taxonomies  falls 
under  a  given  category.  For  this  next  level  of  analysis,  the  sub-codes  were 
translated  into  their  respective  category  within  the  original  tables.  In  DoD  HFACS 
these  categories  consisted  of  organizational  influences,  supervision, 
preconditions,  and  acts.  HFACS-M  changes  the  supervision  category  to 
command.  Participants  1-7  of  Table  9  and  Table  10  were  non-SWOs  and 
participants  8-14  were  SWOs. 
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Table  9.  DoD  HFACS  categories  broken  down  by  Designator/MOS/AFSC 


DOD  HFACS 

Findings 

Number 

Desig/MOS/AFSC 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

1 

49/15 

A 

P 

A 

P 

P 

A 

P 

0 

0 

0 

p 

2 

1310 

P 

P 

P 

P 

P 

A 

P 

0 

0 

0 

p 

3 

0602 

A 

A 

A 

P 

P 

A 

P 

0 

0 

0 

p 

4 

7565 

A 

P 

A 

P 

P 

A 

P 

S 

S 

s 

p 

5 

49A 

P 

0 

P 

P 

P 

P 

P 

0 

0 

s 

p 

6 

21B/49 

A 

A 

P 

P 

S 

A 

P 

0 

0 

0 

0 

7 

1810 

A 

P 

P 

P 

P 

A 

A 

0 

S 

s 

p 

8 

1110 

A 

0 

P 

A 

P 

A 

P 

S 

S 

s 

p 

9 

1110 

S 

A 

A 

A 

0 

A 

0 

0 

0 

0 

0 

10 

1110 

S 

0 

A 

P 

P 

A 

P 

S 

0 

0 

p 

11 

1110 

A 

P 

P 

P 

P 

A 

0 

0 

0 

0 

p 

12 

1110 

A 

A 

A 

A 

P 

P 

P 

S 

S 

s 

p 

13 

1110 

A 

P 

A 

P 

P 

A 

P 

0 

0 

0 

p 

14 

1110 

A 

P 

S 

P 

P 

A 

P 

S 

0 

0 

p 
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Table  10.  HFACS-M  categories  broken  down  by  Designator/MOS/AFSC 


HFACS-M 

Number 

Desig/MOS/AFSC 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

1 

1317 

A 

A 

A 

P 

P 

A 

P 

C 

C 

c 

p 

2 

1810 

A 

A 

P 

P 

P 

A 

P 

C 

0 

0 

p 

3 

0602 

A 

C 

P 

A 

P 

P 

P 

0 

0 

0 

0 

4 

7523 

A 

A 

P 

A 

P 

A 

P 

P 

P 

c 

0 

5 

1310 

A 

P 

A 

P 

P 

A 

P 

0 

C 

c 

0 

6 

1120 

A 

P 

A 

P 

0 

A 

P 

0 

0 

0 

p 

7 

19A 

A 

P 

P 

P 

P 

A 

P 

C 

0 

c 

p 

8 

1110 

A 

P 

P 

P 

P 

A 

P 

0 

0 

0 

p 

9 

1110 

A 

A 

P 

P 

P 

P 

P 

0 

0 

0 

p 

10 

1110 

A 

A 

P 

P 

P 

A 

P 

C 

c 

c 

p 

11 

1110 

A 

P 

P 

P 

P 

A 

P 

0 

0 

0 

0 

12 

1110 

A 

A 

A 

A 

0 

A 

P 

0 

0 

0 

p 

13 

1110 

A 

A 

P 

P 

P 

A 

P 

0 

0 

c 

0 

14 

1110 

P 

P 

P 

P 

P 

A 

P 

0 

c 

c 

0 
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1. 


DoD  HFACS 


From  these  results,  tables  were  constructed  to  calculate  Fleiss’  Kappa. 
Each  category  was  assigned  its  own  column  and  Pj  was  calculated  for  each. 

Four  possible  categories  were  available  in  DoD  HFACS.  Table  11  shows  the 
overall  breakdown  of  categories,  shown  in  the  columns,  and  findings, 
represented  by  the  rows. 


Table  1 1 .  Overall  DoD  HFACS  category  table 


DOD  CAT 

O 

S 

P 

A 

Pi 

1 

0 

2 

2 

10 

0.516 

2 

3 

0 

7 

4 

0.330 

3 

0 

1 

6 

7 

0.396 

4 

0 

0 

11 

3 

0.637 

5 

1 

1 

12 

0 

0.725 

6 

0 

0 

2 

12 

0.736 

7 

2 

0 

11 

1 

0.615 

8 

9 

5 

0 

0 

0.505 

9 

10 

4 

0 

0 

0.560 

10 

9 

5 

0 

0 

0.505 

11 

2 

0 

12 

0 

0.736 

Total 

36 

18 

63 

37 

6.264 

Pj  0.234  0.117  0.409  0.240 


Fleiss’  Kappa  was  calculated  to  determine  inter-rater  reliability  at  the 
categorical  level  for  raters  using  DoD  HFACS.  The  overall  results  were  as 
follows:  P  =  .569;  Pe  =  .293;  K  =  .391. 

Table  11  was  then  divided  between  SWOs  and  non-SWOs.  Fleiss’  Kappa 
was  calculated  for  each  group  individually.  For  SWOs  employing  DoD  HFACS  at 
the  category  level,  the  results  were  as  follows:  P  =  .515;  Pe  -  .272;  K  =  .334. 
For  non-SWOs  employing  DoD  HFACS  at  the  category  level,  the  results  were  as 
follows:  P  =  .619;  Pe  =  .324;  K  =  .436. 
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2. 


HFACS-M 


Fleiss’  Kappa  was  also  calculated  to  determine  inter-rater  reliability  at  the 
categorical  level  for  raters  using  HFACS-M.  Four  possible  categories  were 
available.  Table  12  shows  the  overall  breakdown  of  categories,  shown  in  the 
columns,  and  findings,  represented  by  the  rows. 


Table  12.  HFACS-M  category  table 


M  CAT 

O 

C 

P 

A 

Pi 

1 

0 

0 

1 

13 

0.857 

2 

0 

1 

6 

7 

0.396 

3 

0 

0 

10 

4 

0.560 

4 

0 

0 

11 

3 

0.637 

5 

2 

0 

12 

0 

0.736 

6 

0 

0 

2 

12 

0.736 

7 

0 

0 

14 

0 

1.000 

8 

9 

4 

1 

0 

0.462 

9 

9 

4 

1 

0 

0.462 

10 

7 

7 

0 

0 

0.462 

11 

6 

0 

8 

0 

0.473 

Total 

33 

16 

66 

39 

6.780 

Pj  0.214  0.104  0.429  0.253 


Fleiss’  Kappa  was  calculated  to  determine  inter-rater  reliability  at  the 
categorical  level  for  raters  using  DoD  HFACS.  The  overall  results  were  as 
follows:  P  =  .616;  Pe  =  .305;  K  =  .448. 

Table  12  was  then  divided  between  SWOs  and  non-SWOs.  Fleiss’  Kappa 
was  calculated  for  each  group  individually.  For  SWOs  employing  HFACS-M  at 
the  category  level,  the  results  were  as  follows:  P  =  .645;  Pe  =  .317;  K  =  .481. 
For  non-SWOs  employing  DoD  HFACS  at  the  category  level,  the  results  were  as 
follows:  P  =  .558;  Pe  =  .297;  K  =  .372. 
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Table  13  shows  a  side-by-side  comparison  of  Fleiss’  Kappa  for  both 
HFACS  versions  at  each  of  the  three  levels  analyzed. 


Table  13.  Fleiss’  Kappa  comparison  of  DoD  HFACS  and 
HFACS-M  results  at  all  three  levels 


Overall 

SWO 

NON-SWO 

DOD  HFACS  (Nano-code) 

0.154 

0.114 

0.198 

HFACS-M  (Nano-code) 

0.182 

0.202 

0.137 

DOD  HFACS  (Sub-code) 

0.281 

0.25 

0.293 

HFACS-M  (Sub-code) 

0.355 

0.362 

0.289 

DOD  HFACS  (Category) 

0.391 

0.334 

0.436 

HFACS-M  (Category) 

0.448 

0.481 

0.372 

The  following 


chapter  discusses  the  results  of  this 


analysis  and  their 


implications. 
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V.  DISCUSSION 


A.  DISCUSSION 

Accident  investigations  have  concluded  that  virtually  all  major  mishaps 
that  occurred  within  the  surface  Navy  are  the  product  of  human  error  (Lacy, 
1998).  To  mitigate  or  prevent  mishaps  of  this  nature,  it  is  vital  that  an  appropriate 
method  be  established  to  categorize  and  count  these  errors.  DoD  HFACS  is  one 
method  that  has  been  employed  for  several  years,  but  its  reliability  has  been 
called  into  question  on  more  than  one  occasion.  To  this  end,  a  domain-specific 
version,  HFACS-M,  was  developed  and  tested  against  the  original  version  to 
assess  the  inter-rater  reliability  of  each  instrument. 

B.  RESEARCH  QUESTIONS 

The  goal  of  this  research  was  to  determine  if  the  perceived  domain- 
specific  gaps  in  DoD  HFACS  with  respect  to  the  surface  Navy  could  be  filled  by 
creating  a  maritime  specific  version,  HFACS-M.  This  study  employed  both 
HFACS  taxonomies  in  conjunction  with  a  case  study  to  answer  three  questions. 

1.  Research  Question  #1 

The  first  question  addressed  by  this  study  is:  Do  SWOs  and  non-SWOs 
show  the  same  consistency  when  applying  DoD  HFACS?  HFACS  was  originally 
developed  for  application  in  naval  aviation  mishaps  and  has  been  amended  and 
updated  into  its  current  version,  DoD  HFACS.  The  results  of  this  study  show  a 
slightly  higher  Fleiss’  Kappa  for  non-SWOs  at  every  level  (nano-code,  sub-code, 
and  category)  using  DoD  HFACS,  than  for  SWOs.  Recall  that  Fleiss’  Kappa  is 
used  to  determine  inter-rater  reliability  between  a  given  number  of  raters.  Fleiss’ 
Kappa  indicates  agreement  between  raters  over  that  which  could  be  reached  by 
chance  (Fleiss,  1971).  Fleiss’  Kappa  suffers  from  the  fact  that  it  does  not  have  an 
agreed  upon  measure  of  significance,  primarily  because  the  number  of  subjects 
and  categories  directly  impact  the  value  (Gwet,  2010).  Thus,  it  is  not  possible  to 
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assign  a  particular  meaning  to  a  score  (good,  fair,  bad,  etc.).  However,  it  can  be 
stated  that  the  non-SWOs’  higher  scores  when  using  DoD  HFACS  lead  to  the 
conclusion  that  a  non-domain  specific  taxonomy  yields  a  higher  inter-rater 
reliability  when  employed  with  subjects  not  intimately  familiar  with  the  domain  in 
question  (maritime  in  this  case).  Again,  based  on  the  sample  size  and  the  fact 
that  Fleiss’  Kappa  was  used,  it  is  not  possible  to  say  that  the  difference  was 
statistically  significant.  However,  it  is  clear  that,  in  this  particular  study,  non- 
SWOs  were  more  consistent  when  using  DoD  HFACS.  This  conclusion  supports 
the  findings  of  Wang  et  al.  (2001),  in  which  research  showed  rater  agreement 
below  40%  at  the  categorical  level.  Like  the  2001  study,  this  study  found  that  the 
group  of  raters  applying  DoD  HFACS  had  a  Kappa  of  just  .391,  or  39  %.  It  is 
interesting  to  note  that  when  this  group  was  divided  into  SWOs  and  non-SWOs, 
the  non-SWOs  had  an  inter-reliability  of  .436,  some  10%  higher  than  SWOs  and 
4%  higher  than  the  group  as  a  whole.  Again,  this  result  suggests  that,  when 
faced  with  a  situation  outside  their  scope  of  expertise  (domain),  subjects  have  a 
higher  inter-rater  reliability  using  a  generic  taxonomy. 

2.  Research  Question  #2 

The  second  question  addressed  by  this  study  is:  What  errors,  overlaps,  or 
gaps,  if  any,  currently  exist  in  DoD  HFACS?  Finding  7  from  the  NTSB  accident 
report  deals  with  the  fatigue  experienced  by  the  pilot  of  one  of  the  vessels 
involved.  Nano-code  PC307,  Fatigue  (sleep  deprivation)  was  a  commonly 
selected  response,  but  PC308,  Circadian  rhythm  de-synchronization,  was  also 
chosen  by  some  subjects.  PC308  seems  to  be  a  redundant  code  since  a  de- 
synchronization  of  an  individual’s  circadian  rhythm  causes  fatigue.  Thus,  are 
these  overlaps  within  the  taxonomy?  The  truth  about  overlaps  in  HFACS  seems 
to  be:  It  depends.  It  depends  on  the  person  doing  the  investigation  and  the 
person  assigning  the  codes  based  on  the  finding  of  the  investigation.  The 
wording  of  the  investigation  can  have  considerable  impact  on  which  selection  the 
rater  makes.  Likewise,  the  training  of  the  rater,  along  with  his  or  her  background 
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and  expertise,  all  play  pivotal  roles  in  how  the  rater  perceives  the  situation 
described  in  the  investigation,  and  ultimately,  which  codes  he  or  she  will  select. 

Where  gaps  are  concerned,  DoD  HFACS  does  seem  to  lack  domain 
specificity,  as  asserted  by  Griggs  (2012)  and  Salmon  et  al.  (2012).  Corrective 
and  preventive  maintenance  issues  are  extremely  important  in  all  branches  of 
service,  yet  are  not  a  part  of  DoD  HFACS.  It  should  be  noted,  however,  that  it 
would  be  impossible  to  create  appropriate  nano-codes  for  every  minor  error.  To 
this  end,  domain  specificity  should  not  focus  simply  on  what  is  missing,  but  also 
what  makes  one  domain  different  from  the  next  (i.e.,  the  difference  between 
submarines  and  aircraft),  as  both  can  lead  to  the  discovery  of  gaps. 

Taking  a  broader  look  at  DoD  HFACS  reveals  an  error  classification 
taxonomy  focused  primarily  on  the  event  itself  and  not  necessarily  on  latent 
errors.  This  emphasis  on  proximal  errors  rather  than  distal  ones  tends  to 
eliminate  potential  latent  errors  from  being  identified.  Manufacturing  processes 
that  produce  hardware  and  software  are  less  than  perfect.  Be  it  a  mistake  in  a 
small  string  of  code  or  a  poorly  welded  seam,  these  errors  can  lie  dormant  for  a 
large  portion  of  the  lifecycle  of  a  ship,  aircraft,  or  submarine  until  eventually  the 
exact  series  of  actions  occur  to  cause  them  to  be  revealed  in  a  catastrophic 
manner.  Administrative  processes  that  produce  publications,  instructions,  and 
checklists  are  also  prone  to  error.  The  incorrect  wording  of  an  emergent  action  in 
an  instruction  or  the  incorrect  ordering  of  controlling  actions  for  a  casualty 
situation  has  the  potential  to  cause  more  damage  than  they  prevent.  Issues  such 
as  these  may  be  hard  to  identify  during  the  course  of  an  investigation  and 
impossible  to  quantify  without  being  properly  addressed  in  the  error  classification 
instrument. 

3.  Research  Question  #3 

The  third  question  addressed  by  this  study  is:  Does  a  tailored  version  of 
HFACS  result  in  increased  inter-rater  reliability  when  classifying  mishaps  within 
the  surface  Navy?  Why  or  why  not?  HFACS-M,  the  tailored  maritime  domain- 
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specific  version  of  DoD  HFACS  showed  higher  overall  Fleiss’  Kappa  than  DoD 
HFACS  at  every  level  of  analysis  (see  Table  14).  It  is,  however,  not  a  staggering 
difference.  HFACS-M  had  a  higher  inter-rater  reliability  at  the  categorical  level  by 
5.7%,  7.4%  at  the  sub-code  level,  and  just  2.8%  at  the  nano-code  level.  SWOs 
had  a  higher  inter-rater  reliability  than  non-SWOs  when  using  HFACS-M  at  every 
level.  In  this  study,  SWOs  had  a  higher  inter-rater  reliability  than  non-SWOs  by 
10.9%  at  the  categorical  level,  7.4%  at  the  sub-code  level,  and  6.5%  at  the  nano¬ 
code  level  when  using  HFACS-M.  Fleiss’  Kappa  calculated  for  SWOs  using 
HFACS-M  were  also  higher  than  non-SWOs  using  DoD  HFACS  (by  4.5%,  6.9%, 
and  .04%  at  the  categorical,  sub-code,  and  nano-code  levels,  respectively), 
which  leads  to  the  conclusion  that  subject  matter  experts  (SWOs)  have  a  slightly 
higher  degree  of  agreement  when  using  a  domain  specific  instrument  that 
employs  terminology  with  which  they  are  familiar.  Based  on  the  small  sample 
size  and  untrained  raters,  however,  further  testing  should  be  considered. 

The  conclusion  that  domain  specific  error  taxonomies  produce  higher 
inter-rater  reliability  when  employed  by  subject  matter  experts  appears  to  support 
what  Salmon  et  al.  (2011)  and  Griggs  (2012)  assert,  “the  taxonomy  needs  to  be 
relevant  to  the  maritime  community”  (Griggs,  2012,  p.  85).  In  this  study,  HFACS- 
M,  a  domain-specific  instrument,  resulted  in  a  slightly  greater  overall  inter-rater 
reliability  than  the  more  generic  DoD  HFACS. 
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VI.  CONCLUSIONS  AND  RECOMMENDATIONS 


A.  CONCLUSIONS 

This  research  provides  support  for  what  Griggs  (2012)  asserted;  that 
domain-specific  error  classification  taxonomies,  when  employed  by  experts  in 
that  domain,  may  have  greater  accuracy  than  a  generic  or  non-specific  version. 
Greater  specificity  in  error  classification  leads  to  more  accurate  hazard 
identification,  which  reduces  mishaps  in  both  quantity  and  severity.  This  finding  is 
important  for  the  Navy  and  DoD  as  a  whole  as  fiscal  constraints  set  in  and  yard 
periods  and  dry  dock  availabilities  become  fewer  and  farther  between. 

It  should  be  noted  that  the  study  was  conducted  with  specific  time 
constraints.  The  time  to  research  and  develop  the  apparatus  and  method 
spanned  a  six-month  period.  Although  the  subjects  were  experienced  military 
officers,  none  had  experience  with  HFACS  outside  of  the  brief  training  received 
immediately  prior  to  reading  the  case  study  provided  with  this  research.  Despite 
these  facts,  it  was  still  demonstrated  that  SWOs  using  HFACS-M  displayed  a 
slightly  higher  inter-rater  reliability  than  non-SWOs.  Fleiss’  Kappa  calculated  for 
HFACS-M  was  also  slightly  higher  than  that  of  DoD  HFACS. 

B.  RECOMMENDATIONS 

Based  on  the  results  of  this  study,  future  research  should  address  the 
addition  of  nano-codes  to  address  the  previously  discussed  latent  errors  to 
ensure  a  much  more  robust  taxonomy.  HFACS-M  added  several  nano-codes 
having  to  do  with  maintenance  processes.  Currently,  a  large  gap  exists  in  DoD 
HFACS,  but  both  versions  would  benefit  from  nano-codes  designed  to  account 
for  latent  distal  errors  accurately.  The  development  of  such  codes  would  require 
extensive  study  but  would  add  significantly  to  the  body  of  knowledge  surrounding 
human  error  and  its  classification  and  quantification  within  the  DoD. 
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The  findings  of  this  study  support  the  need  for  domain-specific  human 
error  taxonomies.  However,  the  field  of  human  error  would  benefit  from  more 
extensive  research.  A  study  using  trained  raters  and  a  Naval  Safety  Center 
(NAVSAFCEN)  mishap  investigation  from  the  surface  Navy  could  help  to  validate 
the  HFACS-M  taxonomy. 

Finally,  other  domain-specific  versions  of  HFACS  should  be  developed 
and  studied.  To  assume  all  branches  of  service  and  the  communities  therein 
have  identical  mishap  potentials  is  to  presume  too  much.  The  surface  warfare 
community  is  far  different  from  the  aviation  community,  for  example.  While  they 
can  be  generalized  to  a  degree,  at  some  point,  the  specific  issues  must  be 
identified  in  the  investigation  process.  These  domain-specific  issues  can  then  be 
addressed  so  that  the  number  of  latent  errors  is  reduced  and  the  likelihood  that 
an  unfortunate  chain  of  events  will  lead  to  a  mishap  is  diminished. 
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APPENDIX  A.  HFACS  TRAINING 


Human  Error  Classification 
Training 

LT  Jason  Bilbro 
Human  Systems  Integration 
Naval  Postgraduate  School 


•  Mishaps  cost  the  Department  of  Defense 
billions  each  year 

•  Human  error  is  involved  in  nearly  every  major 
mishap 

•  Mishap  prevention  first  requires  identification 
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•  Hazards  find  pathways  through  defenses  and 
become  losses 

•  Humans  are  not  perfect  and  neither  are  our 
defenses  or  prevention  measures 


•  Shappell  and  Weigmann  created  the  HFACS 
taxonomy  to  turn  theory  into  practice 

•  Four  categories:  Organizational  Influences, 
Supervision,  Precondition,  Acts 

•  Each  category  is  broken  down  into  sub-codes 
with  each  sub-code  into  nano-codes 
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Organizational  Influences 


67 


68 


Example 


•  The  pilot  of  vessel  A,  the  First  Mate,  had  only 
six  hours  of  sleep  in  the  previous  48  hours  due 
to  watch  rotation 

•  The  parent  company  of  vessel  A  encouraged  a 
schedule  that  maximized  profits  and  did  not 
allow  for  proper  crew  rest 

•  The  ship's  master  of  vessel  A  routinely  went 
without  proper  rest  and  did  not  encourage  any 
different  behavior  among  his  crew 

•  The  pilot  of  vessel  B  did  not  adjust  course  in  a 
timely  manner  to  avoid  the  collision. 


•  Carefully  consider  which  of  the  four  levels 
applies. 

-  The  applicable  sub-codes  within  each  level  will 
help  guide  the  nano-code  selection 

•  Many  nano-codes  are  similar.  Pay  attention  to  the 
context  of  the  findings  to  select  the  best  choice. 
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•  Below  are  the  nano-codes  associated  with  Organizational  Influences. 
Take  the  time  now  and  select  the  one  which  best  describes  the 
following: 

—  The  parent  company  of  vessel  A  encouraged  a  schedule  that  maximized  profits 
and  did  not  allow  for  proper  crew  rest 


ORGANIZATIONAL  INFLUENCE 

Nano  Code 

Description 

Resource /Acquisition  Management 

OK  001 

Air  traffic  control  resources  are  deficient 

OR  00? 

Airfield  resources  are  deficient 

OK  003 

Operational  support  facrlities/equipment  are  deficient 

OR  CXVl 

Purshasing  or  providing  poorly  designed  or  unsuitable  equipment 

OR  005 

Failure  to  remove  inadequate/worn-out  equipment  in  a  timely  manner 

OR  006 

Personnel  recruiting  and  selection  policies  are  inadequate 

OR  007 

Failure  to  provide  adequeate  manning/staffing  resources 

OR  008 

T ailure  to  provi  de  adequeate  operational  informational  resources 

OR  009 

Failure  to  provi  de  adequate  funding 

Organizational  Climate 

OC  001 

Organizational  culture  (attitude/ actions)  allows  for  unsafe  mission  demand/pressure 

OC.  00? 

inappropriate  perception  nf  promotion  nr  evaluation  prnredures  lead  tn  an  unsafe  act 

OC  003 

Organizational  over-confidence  or  under-confidence  in  equipment 

Of.  004 

impending  unit  deartivatinn  nr  missinn/equipnent  change  leads  to  unsafe  situatioo 

OC  005 

Organizational  structure  is  unclear  or  inadequate 

Organizational  Process 

OP  001 

Pace  of  ops  icmpo/workl  oad  creates  unsafe  si  tuation 

OP  002 

Organizational  program/ policy  risks  not  adequately  assessed,  leading  to  an  unsafe  situation 

OP  003 

Provided  inadequate  procedural  guidance  or  publications 

OP  004 

Organizational  (formal)  training  is  inadequate  or  unavailable 

OP  005 

Flawed  doctrine/philosophy  leads  to  unnecessary  risks 

OP  006 

i  riadequeaie  program  management  leads  to  unsafe  situation 

^  Organizational  Influences  (cont.) 


•  The  best  selection  in  this  case  is: 

-  OC  001  Organizational  culture  (attitude/actions) 
allows  for  unsafe  mission  demand/pressure 

•  Alternative  selections  for  this  finding: 

OP  001  Pace  of  ops-tempo/workload  creates  unsafe 
situation  is  a  tempting  selection,  but  the  issue  at 
hand  really  speaks  to  an  unsafe  organizational 
climate  that  favors  profits  over  safety. 
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•  Below  are  the  nano-codes  associated  with  Supervision.  Take  the 
time  now  and  select  the  one  which  best  describes  the  following 

-  The  ship's  master  of  vessel  A  routinely  went  without  proper  rest  and  did  not 
encourage  any  different  behavior  among  his  crew 


SUPERVISION 

Nano  Code 

Description 

inadequate  Supervision 

SI  001 

Command  oversight  inadequate 

SI  002 

railed  to  ensure  proper  role-mod elirg 

SI  QOS 

Failed  to  provide  proper  training 

SI  004 

Failed  to  provide  appropriate  policy/guidance 

SI  OOS 

Personality  conflict  with  supervisor 

SI  006 

1  arit  of  supervisory  responses  to  critical  information 

Failure  to  Correct  Known  Problem 

SF001 

Failed  to  identify/correct  risky  behavior 

SF  002 

Failed  to  correct  unsafe  practices 

Planned  inappropriate  Operations 

SP  001 

Directed  mission  beyond  personnel/equipment  capabilities 

SP  002 

Personnel  mismatch 

SP  003 

Selected  individual  with  lack  of  current  OHpcrience 

SP  004 

Selected  individual  with  limited  overall  experience 

SPOCS 

Selected  individual  with  lark  of  proficiency 

SP  006 

Performed  inadequate  risk  assessment 

SP  00/ 

Authorized  unnecessary  hazard 

Supervisory  Violations 

SV  001 

r ailure  to  enforce  existing  rules 

svno? 

Allowing  unwritten  policies  to  become  standard 

sv  oca 

Directed  individual  lo  violate  existing  regulations 

SV  004 

Authorized  unqualified  individuals  tor  mission 

Supervision  (cont.) 


•  The  best  selection  in  this  case  is: 

-  SI  002  Failed  to  ensure  proper  role-modeling 

•  Alternative  selections  for  this  finding: 

—  SF  001  Failed  to  identify/correct  risky  behavior  and 
SF  002  Failed  to  correct  unsafe  practices  are  also 
reasonable  selections,  but  the  finding  lends 
specifically  to  an  issue  with  the  way  the  ship's 
master  allowed  himself  to  be  viewed  and 
emulated. 
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Preconditions 


•  Below  are  the  nano-codes  associated  with  Preconditions.  Take  the 
time  now  and  select  the  one  which  best  describes  the  following: 

—  The  pilot  of  vessel  A,  the  First  Mate,  had  only  six  hours  of  sleep  in  the 
_ previous  48  hours  due  to  the  watch  rotation 


Nano-Code  |  Description 

Physical  Environment 

PE  101 

Id ng/fog  on  window  restricts  vision 

PE  102 

Weather  conditions  restricts  vision 

PE  103 

Vibrations  affect  vision  or  balance 

PE  10d 

Ovst/smoke  tn  workspace  obstructs  vision 

pf  ins 

Windbl.nl  in  war  k-.p.m'  obstruct*.  vision 

PE  106 

Cold  stress 

PE  107 

Heat  st'ess 

PE  103 

Entivmu  lorws  limit  jn  individual's  movement 

pf  irw 

PF  110 

Noise 

PF  11  1 

Brownout  (n./;.,  sand  stnrm)/whiti'out  (r.g,  snow  storm! 

technological  Environment 

SF  001 

Sr.it  and  n-Ji.iinl  svslrms  problems 

sf  no; 

In-.trument.ilinn  .mrl  warning  system  issues 

SF  001 

Visibility  restrictions  [notweatFier  related) 

SF  QW 

Controls  and  switdies  are  inadequate 

srooj 

Automated  system  creates  an  unsafe  situation 

sr  006 

Workspace  Incompatible  with  operation 

Sf  007 

Personal  equipment  interference 

SF  008 

Communi  cat  on  equipment  i  nadeauate 

Self-Imposed  Stress 

PP  201 

Physical  fitness  level  (Inappropriate  formlssion  demands 

PP  202 

Al  cohoi 

PP  203 

Drugs/cver-the-co  unter  medi  cation/suppi e  ments  ( not  presen  be  a  \ 

PP  704 

Nutrition/dirt 

PP  20S 

Inadec  jate  rest  !se]fnmgosedi 

PP  206 

Operatng  with  known  disqualifying  medical  condition 

Coord  i  nati  on/Commun  icanion/Planning  Factors 

PP101 

Failure  of  crew/te  am  leadership 

PP  1IJ7 

H.vlure  in  cross  check/ hark  up 

PP  118 

1  node  quote  task  delegation 

PP104 

Rank/positi on  intimidation 

PP105 

Lack  of  assertiveness 

PP  IDG 

Critical  information  not  commmuni  cated 

PP107 

Stand  ard /proper  terminol  Ofjy  not  u  sed 

PP  ILK 

Faluro  to  ensure  comminiGit ed  in tenlions/arJl ions  were  understood  and  folio 

moo 

Mssion  planning  inadequate 

PP110 

Min  on  briefing  inadequate 

ppm 

Failure  to  re-assess  risk  and  adjust  to  changing  circumstances 

PP  112 

1  nformat)'  on  is  mi  si  nterpreted  or  di  sregarde  d 

Awareness  (Cognitive)  Factors 

PC  101 

N  ft*  payi  ng  attention 

pc  w 

Fi*ation ("channel  i  rod  attention") 

pc  ice 

Task  over-satu'ation  (e.g.,  too  much  inf  cwmation  to  process) 

PC  104 

Confusion 

PC  105 

Negative  transfer  !e.g..  using  old  proceduresfornew  system) 

PC10G 

D'str  action 

pc:  iu/ 

Geographically  lost  (confusion  about  location) 

PC  ILK 

Interference/i nterrupb on  dunng  task 

Physical/Menta’  Limitations 

PC 401 

Learning  rate  limitations 

PC  4  02 

Memory  limitations 

PC  4  03 

Body  size/movement  limitations 

PC  404 

Coordination  deficiency 

PC  405 

Technical  or  procedural  knowledge  not  retained  after  training 

‘List  of  Preconditions  continued  on  next  page* 
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Preconditions 


•  Below  are  the  nano-codes  associated  with  Preconditions.  Take  the 
time  now  and  select  the  one  that  best  describes  the  following: 

—  The  pilot  of  vessel  A,  the  First  Mate,  had  only  six  hours  of  sleep  in  the 
previous  48  hours  due  to  watch  rotation 


Pi'f  unptu.il  F.uinrs 

PCVM 

rvhtinn  illusion 

PC  502 

l  jrni  ng  4  lusion/baiance 

PC  503 

FCSM 

Misperception  of  changing  environment 

PC  505 

Misinterpreted/misread  Instrumentle.g.,  misjudge  a'titude/dl  stance /speed} 

PC  506 

Inna  curate  expectation  (e.g..  seeing/hearing  what  is  expected  instead  of  what  is  actually  tnere/heai 

PC  50/ 

Misinterpretation  of  auditory  cues 

PC  508 

Spatial  disorientation  -  not  recognized 

PC  500 

Spatial  disorientation  -  recognized 

PC  510 

Spatial  disorientation  -  Incapadtatlng 

PC  511 

Time  distortion 

PC  201 

Pre-existing  personality  disorder 'professionally  diagnosed) 

PC  701 

PC  704 

Filiation, X  -.l.iti- 

PC  70S 

Personality  -,t>  r 

PC  206 

Overconfidence 

PC  707 

Pr •  •  pushingsrll  or  I’tiuipinnnt  Irn  h.ari] 

PC  708 

PC  209 

Not  enough  motivation 

PC  210 

Misp:aced  motivation 

PC  211 

More  aggressive  than  necessary 

PC  212 

Excessive  motivation  to  succeed  [e.g„  "do  or  die") 

PC  211 

"Get-home-it-is"/"get-there-it-is" 

PC  214 

Inappropriate  response  due  to  expectation 

PC  215 

Motivational  exhaustion  ("burnout") 

Adverse  1‘sysiologi  cal  btress 

PC  301 

effects  of  G  forces  (e.g..  G-LOC) 

pc:  xv 

Effects  of  prescribed  drugs 

PC3CB 

Operational  mjury/illness 

PC  304 

Sudden  incap  acirati on/uncon scion  sn  ess  ( not  due  to  fi) 

PC  306 

Pre  existing  physical  illness/miurv 

PC  305 

Physical  overexertion 

PC  307 

Fatigue  (sleep  deprivation) 

PC  308 

Orcadian rhvthmde  synchronization  (e.g.. »et  lag or  sliifi  work) 

PC  300 

Motion  sickness 

PC  310 

Trapped  gas  disorders 

PC  3)1 

rvolved  gas  disorders  (e.#.,  decompression  siekness/hends) 

PC  312 

Reduced  oxygen  (hypoxia) 

PC  313 

llyperventilation{rapid  breathing) 

PC  314 

Inadequate  adaptation  to  darkness 

PC  315 

Dehydration 

PC  31G 

Physical  task  over- saturation 
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•  The  best  selection  in  this  case  is: 

-  PC  307  Fatigue  (sleep  deprivation) 

•  Alternative  selections  for  this  finding: 

-  PP  205  Inadequate  rest  (self-imposed)  seems  like  a 
good  choice,  but  it  was  the  watch  rotation  that 
took  its  toll,  not  a  direct  decision  by  the  First  Mate 

•  This  finding  deals  directly  with  adverse  physiological 
stress,  in  this  case  fatigue,  experienced  by  the  First  Mate. 


•  Below  are  the  nano-codes  associated  with  Acts.  Take  the  time  now 
and  select  the  one  that  best  describes  the  following: 

—  The  pilot  of  vessel  B  did  not  adjust  course  in  a  timely  manner  to  avoid  the 
collision 


Nano  Code 

Description 

Skill-Based  Errors 

AF  1G1 

Unintended  operation  ot  equipment 

AE  102 

Checklist  not  followed  correctly 

af  im 

Procedure  not  followed  correctly 

At  104 

Over  Controlled/undcr  controlled  aircraft/vehide 

AE  105 

Breakdown  in  visual  scan 

AFlOTi 

Inadequate  Anti-G  straining  maneuver 

Judgement  and  Decision-Making  Frrors 

AF201 

inadequate  real-time  risk  assessment  (e.g.,  failure  of  time-critical  ORM) 

At  202 

Failure  to  prioritise  tasks  adequately 

AE2G3 

Rushed  a  necessary  action 

AF204 

Delayed  a  necessary  action 

AF7CK 

Ignored  a  rautior/warning 

AF  706 

Wrong  choice  of  action  during  an  operation  (e.g.,  response  to  an  emergency) 

Perception  Errors 

AE301 

Incorrect  response  to  a  misperception  (e.g.,  visual  illusion  or  spatial  disorientation) 

Violations 

Avnm 

Work-around  violation  (e.g.,  breaking  the  rules  is  preeiveri  as  the  best  solution) 

Avon? 

Wiriespread/routine  violation  (e.g.,  habitual  deviation  from  the  rules  that  is  tolerated  by  managemer 

AV  003 

Extreme  violation  (e.g.,  a  violation  not  condoned  by  management 
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•  The  best  selection  in  this  case  is: 

-  AE  204  Delayed  a  necessary  action 

•  This  finding  deals  directly  with  judgment  and  decision¬ 
making.  In  this  case,  the  failure  of  the  pilot  to  act  in  a 
timely  manner  directly  caused  the  mishap 
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APPENDIX  B.  HFACS-M  TRAINING 


Human  Error  Classification 
Training 

LT  Jason  Bilbro 
Human  Systems  Integration 
Naval  Postgraduate  School 


•  Mishaps  cost  the  Department  of  Defense 
billions  each  year 

•  Human  error  is  involved  in  nearly  every  major 
mishap 

•  Mishap  prevention  first  requires  identification 
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•  Hazards  find  pathways  through  defenses  and 
become  losses 


•  Humans  are  not  perfect  and  neither  are  our 
defenses  or  prevention  measures 


DOD  HFACS 


•  Shappell  and  Weigmann  created  the  HFACS 
taxonomy  to  turn  theory  into  practice 

•  Each  category  is  broken  down  into  sub-codes 
with  each  sub-code  into  nano-codes 

•  HFACS-M  looks  at  Organizational  Influences, 
Command,  Preconditions,  and  Acts 
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Organizational  Influences 
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Example 


•  The  pilot  of  vessel  A,  the  First  Mate,  had  only 
six  hours  of  sleep  in  the  previous  48  hours  due 
to  watch  rotation 

•  The  parent  company  of  vessel  A  encouraged  a 
schedule  that  maximized  profits  and  did  not 
allow  for  proper  crew  rest 

•  The  ship's  master  of  vessel  A  routinely  went 
without  proper  rest  and  did  not  encourage  any 
different  behavior  among  his  crew 

•  The  pilot  of  vessel  B  did  not  adjust  course  in  a 
timely  manner  to  avoid  the  collision. 


•  Carefully  consider  which  of  the  four  levels 
applies. 

-The  applicable  sub-codes  within  each  level  will 
help  guide  the  nano-code  selection 

•  Many  nano-codes  are  similar.  Pay  attention  to  the 
context  of  the  findings  to  select  the  best  choice. 
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•  Below  are  the  nano-codes  associated  with  Organizational  Influences. 
Take  the  time  now  and  select  the  one  which  best  describes  the 
following: 

-  The  parent  company  of  vessel  A  encouraged  a  schedule  that  maximized  profits 
and  did  not  allow  for  proper  crew  rest 


Nano-Code 

Description 

Resource/Acquisition  Management 

Oft  001 

Port  facilities  are  deficient 

Oft  002 

Channel  markers/I ighting are  deficient 

IJRCXK 

opemtion.il  support  f.vjhties/equipment.ve  deficient 

OROoa 

Purshasing  or  providing  poorly  designed  or  unsuitable  equipment 

(MOOS 

hvl  urn  to  ivoatrv  now  svstems/upgr.idesm  a  timel  v  manner 

UftOOb 

failure  to  remove  madequate/wom-out  equipment  .n  a  timely  manner 

Oft  007 

Personnel  recruitingand  selection  oolio'es  are  inadeauate 

lift  IKK 

►  .>111110  to  provide  .idequf.ite  manning/staffing  resources 

Oft  000 

Failure  toprovide  adequeate  operational  informational  resources 

OR  010 

Fail  urn  to  provide  adequate  fundi  np 

lift  Oil 

In.id  equate  quality. issumnce  within  rn.nnt.ip.vice  activity  leads  lounsafe  situation 

Organizational  Climate 

IXUOl 

llrp.viuation.il  culture  (attitude /actions!  allows  for  unsafe  mission demaod/pressure 

OC  002 

Inappropn  ate  perception  of  promotion  or  evaluation  procedures  lead  to  an  unsafe  act 

OCOCG 

Organizational  over-confidence  or  under-confidence  in  equipment 

oc  ixu 

Impending  unit  deactivation  or  mission/equipment  change  loads  to  unsafe  situation 

OC  00G 

Organizational  structure  is  undear  orinadequate 

Organizational  Process 

or  001 

Pace  of  ops-tempo/workload  creates  unsafe  situation 

or  002 

Organizational  program/policy  risks  not  adequately  assessed,  leading  to  an  unsafe  situation 

lift  IKK 

Provided  inadequate  procedural  guidance  or  puhl irations 

otcxm 

Org an i zati on ai  (formal H rain  ing  is  inad  ea u  ate  or  u navai  1  abl e 

or  005 

Flawed  doctrine/philosophy  leadstounnecessary  risks 

or  006 

Inadequeate  program  management  leadsto  unsafe  situation 

or  007 

Organizational  process pnovides inadequate,  untimely  guidance 

•  The  best  selection  in  this  case  is: 

—  OC  001  Organizational  culture  (attitude/actions) 
allows  for  unsafe  mission  demand/pressure 

•  Alternative  selections  for  this  finding: 

-  OP  001  Pace  of  ops-tempo/workload  creates 
unsafe  situation  is  a  tempting  selection,  but  the 
issue  at  hand  really  speaks  to  an  unsafe 
organizational  climate  that  favors  profits  over 
safety. 
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Command 


•  Below  are  the  nano-codes  associated  with  Command.  Take  the 
time  now  and  select  the  one  which  best  describes  the  following 

-  The  ship's  master  of  vessel  A  routinely  went  without  proper  rest  and  did  not 
encourage  any  different  behavior  among  his  crew 


Nano  code 

Description 

Inadequate  Supervision 

a  00 1 

Inadequate  oversight 

a  oop 

Failed  to  ensure  proper  role  modeling 

cioce 

Failed  to  provide  appropriate  poll  cv/gui  dance 

a  cm 

Personality  conflict  with  supervisor 

aoos 

Lack  of  supervisoryrespoosesto  critical  information 

a«* 

Failed  to  communicate  intent <e.g..  standi  ngorders/mtfit  orders! 

Failure  to  Correct  Known  Problem 

CF  001 

Failed  to  identify/ correct  risky  behavior 

C>  lit? 

Failed  lo  correct  unsafe  practices 

Mannmg/Personnel/Trammglssues 

CM  001 

Directed  mission  beyond  oersonnel/eou'pment  capabi  ities 

CM  lit? 

Personnel  mismatch 

CM  003 

Selected  individual  with  lack  of  current  experience 

CM  004 

Selected  individual  with  limited  overall  experience 

CM  005 

Selected  individual  with  lack  of  proficiency 

CM  006 

Directed  mission  without  sufficient  manning 

CM  007 

Com  mand  (formal )  trai  ni n  g  i  nad  eq  uate 

CM  008 

Authorized  unqualified  individualsfor mission 

Wanned  Innappropnate  tlper.it ions 

CT  001 

Authorized  unnecessary  hazard 

CP  007 

Performed  i nadequate  risk  assessment  (ORM) 

Supervisory  Violations 

CVOOl 

Failure  to  enforce  existing  rules 

CV  002 

Allowing  unwritten  pol  ides  to  become  standard 

CV  003 

Directed  individual  to  violate  existing  regulations 
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•  The  best  selection  in  this  case  is: 

-  Cl  002  Failed  to  ensure  proper  role-modeling 

•  Alternative  selections  for  this  finding: 

—  CF  001  Failed  to  identify/correct  risky  behavior  and 
CF  002  Failed  to  correct  unsafe  practices  are  also 
reasonable  selections,  but  the  finding  lends 
specifically  to  an  issue  with  the  way  the  ship's 
master  allowed  himself  to  be  viewed  and 
emulated. 


•  Below  are  the  nano-codes  associated  with  Preconditions.  Take  the 


time  now  and  select  the  one  which  best  describes  the  following: 


-  The  pilot  of  vessel  A,  the  First  Mate,  had  only  six  hours  of  sleep  in  the 
previous  48  hours  due  to  the  watch  rotation 


CnfN  riin.it  irin/Cnmmiinic.ilirin/PI. inning  Factor*. 

ppioi 

r  allure  ol  watcnteam/crew  leadership 

FP  102 

Failure  to  cross  ghudr/bjck  up 

ppiitc 

In.idi'qu.ili-  task  di'li'g.ili  nn 

PP1H4 

Rank/pn'.itinn  intimidation 

PP  ll>» 

■  ark  of  assnrtivnnrM 

PP  lOt 

Critical  information  not  commmunicated 

PP  10/ 

Standard/proper  terminology  not  used 

YY  108 

f.iilutr  In  1'iv.nri'  communicated  inlenlirirv./ailimv.  were  unrfcM'.Irmd  and  followed 

YY  109 

Mission  planning  inadeauate 

YY  no 

Mssion  briefing  inadequate 

YY  111 

failure  to  re-assess  risk  and  adjust  to  changing  circumstances 

PP  112 

Information  Is  misinterpreted  or  disregarded 

Awareness  (Cognitive}  factors 

PC  101 

N  ot  pavi  ng  attention 

PC  102 

Fixation  ("channelized  attention"} 

PC  103 

Task  over-saturation  (e.g..  too  much  Information  to  process} 

pcioa 

Confusion 

PC  105 

Negative  transfer  |e.g„  using  old  procedures  tor  new  system} 

pc  ins 

Detraction 

PC  1D7 

CiiTif;i.i|]hiiMlly  ln*.t  («>nl u'.inn  .ilinut  location} 

PC  106 

Interference/Interruption  during  task 

P1wsicjl/S*L‘ntjl  linuLitiorn 

PC  401 

li-.irnin>:  tall'  limit,  it  ion1. 

PC  407 

Memory  limitation*. 

PC  403 

Forty  si  m/movement  limitations 

PC  40-1 

Coordination  defio  ency 

PC  405 

iechnical  or  procedural  knowledge  not  retained  after  training 

‘List  of  Preconditions  continued  on  next  page* 
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•  Below  are  the  nano-codes  associated  with  Preconditions.  Take  the 
time  now  and  select  the  one  that  best  describes  the  following: 

—  The  pilot  of  vessel  A,  the  First  Mate,  had  only  six  hours  of  sleep  in  the 
previous  48  hours  due  to  watch  rotation 


_ Perceptual  factors _ 

Adverse  PsysrolORical  stress 

prsoi 

Italian  lllu-.ion 

PC  301 

Effects  of  prescribed  drugs 

peso? 

Turning  illu'.inn/b.il.ino' 

PC  302 

Operational  injury/illness 

pc  sot 

PC  303 

Sudden  incapadtabon/unconsoousness 

PC  'XM 

itaperreptinn  of  changing  environment 

PC  354 

Pre  C'*i sting  physical  illness/iniuiy 

pesos 

Mi  sirterp'eted/misread  instrument  (e.g.,  mi  sfudge  altitude/distance/speed) 

PC  306 

Physical  owerpxerlion 

prsnti 

Inn.iiur.ili-  i-xps-ii.itirm  (i-.g.,  MM'ing/hiMrirtK  wh.it  is  I'npriii-rl  iri-.li'.irl  r»1  wh.it  i-.  .iiiu.illy  thi-rr/hi-.irrl 

PC  306 

pc  'Xu 

Msinterpretatinn  of  aurito^ije^^ 

PC  307 

Circadian  rhythm  de-synchrom  tab  on  (watch  rotation  or  shift  work) 

pc  see 

Spatial  disorientabon  •  notrecogmed 

PC  308 

Motion  sickness 

PCS09 

spatial  disorientabon  -  recognited 

PC  309 

PC  S10 

Spatial  dlsorientabon-lncapadtabng 

PC  511 

Time  distortion 

PC  311 

Inadequate  adaptation  to  darknett 

- ^77^ - 

PC  312 

Dehydration 

PC  202 

Pre-existing  psychological  disorder < professionally  diagnosed) 

PC  313 

Physica1  task  over- saturation 

PC  203 

Pre-exlstlngpsychosodai  problem  (professional! v  diagnosed] 

PC2M 

Emotional  state 

pr  70s 

Personality  style 

PC  207 

PressingCe.g..  pusnogself  or  equipment  too  haro] 

PCTfS 

Cnmpl.m-ncy  .ib-.ifice  nl  wuiryl__ 

pr?w 

Not  enough  motivation 

PC  210 

M-.pl.m-rl  motivation 

PC  211 

Itare  aggressive  than  necessar^ 

PC  212 

Excessive  mobvabon  to  succeed  (e.g.,  "do  ordie|2 

PC  213 

"Gel  hnmr  il  isVRet  there  it  is” 

pr  no 

Inappropriate  response  due  to  exportation 

PC  215 

Motivational  exhauston  ("burnout") 

•  The  best  selection  in  this  case  is: 

-  PC  306  Fatigue  (sleep  deprivation) 

•  Alternative  selections  for  this  finding: 

-  PP  205  Inadequate  rest  (self-imposed)  seems  like  a 
good  choice,  but  it  was  the  watch  rotation  that 
took  its  toll,  not  a  direct  decision  by  the  First  Mate 

•  This  finding  deals  directly  with  adverse  physiological 
stress,  in  this  case  fatigue,  experienced  by  the  First  Mate. 
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•  Below  are  the  nano-codes  associated  with  Acts.  Take  the  time  now 
and  select  the  one  that  best  describes  the  following: 

-  The  pilot  of  vessel  B  did  not  adjust  course  in  a  timely  manner  to  avoid  the 
collision 


Nano  Code- 

Description 

Skill  Bosed  Errors 

AE  101 

Unintended  operation  of  equipment 

AF  10? 

Checklist  not  followpd/not  followed  correctly 

AF  103 

Procedure  not  fnllowed/not  followed  correctly 

At  104 

Over  Controlled  or  under  controlled  vessel 

At  105 

Breakdown  in  visual  scan 

Judgement  and  Decision  Making  Errors 

AF?01 

Inadequate  real-time  risk  assessment  (e.g..  failure  of  time-critical  ORM) 

AF  ?0? 

Failure  to  prioritize  tasks  adequately 

AE  203 

Rushed  a  nece ssar^acticn 

AE  204 

Delayed  a  necessary  action 

AE  205 

Ignored  a  caution/waming 

AF  ?06 

Wrong  choice  of  action  during  an  operation  (e.g.,  response  to  an  emergency) 

Perception  Errors 

AE  301 

Incorrect  response  to  a  misperception  (e.g..  visual  illusion  or  spatial  disorientation) 

Violations 

AV001 

Work-around  violation  (e.g.,  breaking  the  mips  is  preeived  as  the  hpst  solution) 

AV00? 

Widpspread/routine  violation  (p.g..  habitual  deviation  from  the  rules  that  is  tolerated  by  management) 

AV  003 

Extreme  violation  (e.g..  a  violation  not  condoned  by  rn  unagement^ 

•  The  best  selection  in  this  case  is: 

-  AE  204  Delayed  a  necessary  action 

•  This  finding  deals  directly  with  judgment  and  decision¬ 
making.  In  this  case,  the  failure  of  the  pilot  to  act  in  a 
timely  manner  directly  caused  the  mishap 
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APPENDIX  C.  HFACS  (EXCEL) 


Perceptual  Factcc* 

2 

4 

6 

13 

PC  501 

Motion  illusion 

PC  502 

Turning  flluslon/balance 

PC  503 

Visual  Illusion 

PC  504 

M'tper  caption  ol  (hanging  environment 

PC  505 

M'sinterpretedimisreed  Instrument  (e.g.,  mlsfudgealtioude/distence/speed) 

PC  506 

inneajratc«pcctation(c.g.,  secing/hcaring  whacls«pectcd  instead  of  what  is  actually  ther^hcard) 

PC  507 

MMnterpr etation  of  auditory  cue* 

PC  503 

Spatial  disorientation  •  not  reccgrYzed 

PC  509 

Spatial  disorimtatfon  recognized 

PC  510 

V^Bal  disorientation  -  fryapadtatlng 

PC  511 

Time  detection 

P*ycho-6ehwvior«<  Factor* 

Pre^dstlng  personality  dsorder  (professionally  (Jagnosed) 

PC  202 

Pro  -ousting  psychological  dsorder  (professionally  dagnosed] 

Pre^d sting  psytfiosodal  problem  (professionally  diagnosed) 

PC  204 

Emotional  state 

Personality  style 

PC  206 

Overconfidence 

Pressing  (eg.,  pushing  sdf  or  equipment  too  hard) 

Complacency  («•«.,  absence  of  worry) 

PC  209 

Not  enough  motivation 

■V --placed  motivation 

More  aggressive  than  necessary 

Excestive  motfvatfon  to  succeed  (eg.,  "do  or  del 

"c>t-home4t-(»’/’get-th«eHtH** 

PC  214 

i  nappr operate  response  due  to  expectation 

PC  215 

Motivational  exhaustion  {turnout"] 

Adverse  PsysiologloJ  Strew 

PC  501 

Effects  of  G  forces  (eg-.  G  LOC) 

PC  302 

Effects  of  preserved  drugs 

PC  303 

Operational  irfury/niness 

PC  304 

Sudden  incapao'tation/unconsctousness  {not  due  to  G) 

PC  305 

Pre-rtt  Mingphyiac*  ilksexx/injury 

PC  306 

Physical  over  exertion 

PC  307 

Fatigue  (sleep  deprivation) 

PC  3C« 

Circadian  rhythm  de-synchronization  (e.g..  jet  or  shift  work) 

PC  309 

Motion  sickness 

PC  310 

Trapped  gat  disorder* 

PC  311 

Evolvedgas  d see  decs  (eg.,  decompression  sickness/bends) 

PC  312 

Reduced  oxygen  (hypoxia) 

PC  313 

Hyperventilation  (rapid  breathing) 

PC  314 

Inadequate  adaptation  to  darkness 

PC  315 

Dehydration 

PC  316 

Physical  task  over -saturation 

SUPERVISION 

Nano-Code  |  Description 

Inadequate  Supervision 

3  001 

Command Oversfeht  inadequate 

SJ  002 

F  niled  to  ensure  proper  role-modeling 

SI  003 

Failed  to  provide  proper  training 

a  004 

SI  005 

Personality  conflict  with  supervisor 

3  006 

Lack  of  supervisory  responses  to  critical  information 

Failure  to  Correct  Known  Problem 

SF  001 

Failed  to  fdcntffy/correct  risky  behavior 

SF  002 

Failed  to  correct  unsafe  practices 

Planned  inappropriate  Operations 

SP001 

Oircctcd  mission  beyond  personnd/cqgipment  capabilities 

SP002 

P«  ver-rvd  mnmitdi 

SP  003 

Selected  intfvidual  with  lack  of  current  experience 

SP004 

Sxiocted  irvf victual  with  limited  overall  experience 

SP005 

SHected  Individual  with  lack  of  proficiency 

SP  006 

Performed  inadequate  risk  assessment 

SP007 

Au  thoc  ized  u  nnecewar  y  hazard 

Supervisory  violations 

SVQQ1 

Failure  to  enforce  existing  rules 

v  ■  ; 

Allo-wfrig  unwritten  polides  to  become  standard 

SV003 

Oircctcd  InJvIdual  to  violate  existing  regulations 

SV004 

Authorized  unqualified  individual*  for  mi*xion 

ORGAN  BATlONAl  INFLUENCE 

Nano  Cod®  |  Description 

Resour  C«f Acquisition  Management 

OR  001 

Air  traffic  control  resourcesaredeficlent 

OR  002 

Airfield  reiouro e*  are  defldent 

OR  003 

Operational  support  fadUties/equipment  aredeffdent 

OR  004 

Purchasing  or  providing  poorly  designed  cr  unsuitable  equipment 

OR  005 

Follute  to  remove  Inadequate/worn-out  equ'prnent  In  a  timely  manner 

OR  006 

Per  vennd  recruiting  and  selection  policies  are  Inadequate 

OR  007 

Failure  to  provide  wdequeate  ma  mlng/staffing  resource* 

OR008 

f  allure  to  provide  adequeate  operational  informational  resources 

OR  009 

Failure  to  provide  adequate  funding 

Organizational  Climate 

OCOOl 

Organizational  culture  (attftude/actions)  allows  for  unsafe  mission  demand/pressure 

OC  002 

t  n-jppt opr ut<*  perception  of  promotlonor  evaluation  procedures  lead  to  an  unsafe  act 

OC  003 

Organizational  over -confidence  or  under -confidence  In  equipment 

OC  004 

impending  unit  deactivation  cr  mfsslon/cquipment  change  leads  to  unsafe  situaticn 

OC  005 

Organizational  structure  is  undear  or  inadequate 

Organizational  Process 

OP  001 

Paw  of  opi  tempc/workload  create*  unsafe  situation 

OP  002 

Organizational  progrem/policy  risks  not  adequately  assessed,  leadng  to  an  unsafe  situation 

OP  003 

Provided  inadequate  procedural  guidance  or  publications 

OP  004 

Organizational  (fcrmal)  training  is  Inadequate  or  urwvwlable 

OP  005 

Flawed  doctrine/phllosophy  leads  to  unnecessary  risks 

OP  006 

inadequate  program  management  lead*  to  unsafe  situation 
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APPENDIX  D.  HFACS-M  (EXCEL) 


Pisu'ptunl  Fuel  or  ■, 

1 

: 

3 

4 

5 

fi 

7 

j? 

9 

10 

PC  SOI 

Mcil inn  illij-.lcm 

lurnine  illusion/balance 

pesos 

l/i'jiiil  illii'iori 

PC  504 

Misperception  of  charging  environment 

PC  see 

Misinterpreted/misread  instrument  (eg.,  misiudge  altitude/dstance/s£eed] 

PCbOO 

inn acurate  expectation  (eg.,  seeing/hearing  what  is  expected  n stead  of  what  is  actually  there/heard) 

PCS07 

Misinterpretation  of  audtorycues 

pesos 

Spiitcil  ti-zuniil.tton  nut  (ifiiy.nimi 

PC  SOO 

spat  al  dsonentat'on  -  recognized 

PCS10 

3fiiil-.il  d-xirirnlitHin  iiMsip.iiihitiry. 

PC  ill 

It  me  distortion 

Psycho  Behavioral  Factors 

PC  201 

Preexisting  personality  disorder  (professionally  diagnosed) 

PC  202 

Pie-existirK  psychological  disorder  (pr  ol  essi  on  ally  ti  ayric^edj 

PC  203 

Pre-existing  psychosocial  problem  (professionall£diagnose2i 

PC  204 

Cniotional  state 

PC  2U» 

Personality  stylo 

PC  206 

Overconfidence 

PC  207 

P ressinp.  (ag,  pushing  sd#  nr  eq upmpnt  too  h  ard) 

PC  208 

Complacency  (ee..  absence  of  worrv| 

PC  209 

Not  enough  motivation 

PC  210 

Misp’aced  motivation 

PC  211 

PC  212 

excessive  motivation  to  succeed  |e.g.,  'do  or  die") 

PC  213 

*Grt  harm*  it  !•.*/'««■*  INn-il  I*.’ 

PC  214 

inaporoonate  response  due  to  expectation 

PC  21S 

M.tr)  rv.ilicm.il  i-xhidi-.llcin  ["burnout*) 

Adverse  Psysiological  stress 

PC  301 

effects  of  prescribedjjrugs 

PC  302 

Uperational  iniury/illness 

PC  303 

Sudden  incapacitation /unconsciousness 

PC  304 

Pri‘iKi-Jiry  piiy-jisi  ilbii'v./injury 

PC  3CS 

Physical  overexertion 

pc:  306 

Fiil^ui*  ('Jirp  tkfiriviiduri) 

PC  30/ 

circadian  rhythm  de- synchronization  (watch  rotation  or  shift  work) 

PC  306 

Motion  sickness 

PC  311 

Reduced  oxygen  (hypoxia) 

PC  312 

Hyperventilation  (rapid  breathirij^^ 

PC  313 

inadequate  adaptation  to  darkness 

PC  314 

Dehydration 

PC  Hill 

i‘ hync.il  task  ovrr  -.itur.ihon 

COMMAND 

Nano-code  |  Uesmr*io^^^ 

Inadequate  Supervision 

SI  001 

inadequate  oversight 

SI  002 

railed  to  ensu  re  proper  role-m  odeli  ng 

SI  003 

F.iiltd  to  ivrzvidr  ithiht  huiruri^^^^ 

SI  004 

r ailed  to  provide  apprepnate  policy/guid80ce 

SI  Offs 

PiT-jrri.lily  mnllici  with  'jipivvr.rir 

SI  000 

Lack  of  supervisory  responses  to  critical  information 

SI  007 

Fiiilitl  to  rmimuriimti-  intent  (i-y,.r-.liiiHfaiR  or di-t^night  ijicIit-.) 

Failure  to  correct  Known  Problem 

srcoi 

railed  to  identify/correet  risky  behavior 

SF  002 

Failedto  correct  unsafe  practices 

Manning/Personnel/Trainirg  Issues 

sp  cni 

Dwi'iti-dmiviiTr  tri'yanii  im -r -xiriti i^/«tr-i iinrien t  nifxibilitn-. 

SP002 

Personnel  mismatch 

SI*  <XX3 

uHr 'tlrdindiviilu.il  with  l.uk  of  iiirrtnl  r*pi  rimer 

SP001 

3 elected  l  ndwl  du  al  with  limited  overall  experience 

SI*  (XI. 

>rtrctrd  mrtivirli.il  with  l.vJc  of  praf icirncy 

SP006 

Directed  mission  without  sufficient  mannlrg 

SP  007 

Command  (lormal)  training  irrudecjuate 

SP003 

Authonzed  unqualified  mdvidualsfor  mission 

Planned  Iririupixoprlulo  Operations 

SP  101 

Authonzed  unnecessary  hazard 

SP  102 

PiTltinrusiui.itliSHi.4i'  ri'*  irv.iv.rntnl  (fiRM) 

Supervisory  Violations 

SV001 

railureto  enforce  existing  rules 

SV002 

Allowing  unwritten  policies  to  become  standard 

SVC03 

Directed  Individual  to  vio'ate  existing  regulations 

OR  (VAN  l/A  Tl  0  N  A 1  IMFI IIFNCT 

Nano-Code  |  Description 

R«-.nj r i:i ■/ Acz|ii i*.it inn  MiiM;y-;<irirnl 

UK  001 

Port  fac  'ties  are  deficient 

OR  (W? 

Chinni'l  iriivkis'./llghling  iircdHicmil 

UK  003 

uperational  support  facihties/equipment  are  deficient 

CKO04 

Purchas-rg  or  providirg  poorly  designed  or  unsuitable  eq.i  pment 

OR  010 

Fiiilurctn  proartrurw  '.y.lisnv/urvriiik". In  .i  lirmiviriiinniT 

CK  CCD 

railureto  remove  inadequate/wcm-out  equipment  in  a  timely  manner 

UK  OOts 

PenonriH  riszmlirK.mil  -aHisliCNi  pcrhcli-.  .in- irmd^jiuiSi^^^ 

CK  007 

Tailureto  provide  adeiiueaternanrnng/itatllrig  resources 

(KOOK 

Fiiiluri'to  proviik'  i(lis|i<isdi'0|S'rail(vii4  mfofiruituin.il  n-aiurti-. 

CKOCO 

Failureto  provide  adequatefundng 

CK  010 

Inadequate quah tv  assurance  within  iriainlanance  activity  leads  to  unsale  situation 

Organizational  Climate 

OCCOl 

Organizational  culturofuttitudu/uctlonsl  a’ Tows  lor  unsafcmission  dijnansi/|xcviur^^ 

cx:txi7 

inappropriate  perception  nf  promrstian  nr  evaluation  procedures  lead  to  an  unsafe  act 

0CC03 

Organizational  over  corilidcncc  or  under  conlidcsicv  in  equipment 

cx:ocm 

impenclnp.  unit  deactivation  or  mission/eqiipment  change  leads  to  unsafe  seuahon 

neons 

fjrgiaiiziihiTi.il  '.liuiiiiri-r.  unriivir  cm  auidisiiiili' 

Organizational  Process 

OPCOl 

UP  002 

organizational  program/pclicy  nsks  no:  aoequateiv  assessed  leading  to  an  unsafe  situation 

0PCO3 

Provided  inadequate  procedural  guidance  or  oub'jcations 

0  PC-04 

nrgiariziitiixi.il  (Iniriiiil)  Ir.irarv  I'.  iiiiiik'C*iiiti*or  uniNiiiLiblc* 

0PCC6 

flawed  doctrine/ philosophy  lead >  to  unnecessary  risks 

OP  006 

lriadis|uiciti'piCKriirn  mu>i;w<riiTlI  bird.  Ici  unvifi*  situation 

UP  00/ 

urganizaticnal  process  pr<wrdesmadecKiate,untimely  guidance 
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APPENDIX  E.  THESIS  DATA 


A.  NANO  CODE 


DOD  HFACS 

Findings 

Number 

Desig/MOS/AFSC 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

1 

49/15 

AV001 

PP103 

AE202 

PP106 

PC307 

AE104 

PC  307 

OP003 

OP006 

OP002 

PE  204 

2 

1310 

PP103 

PP101 

PC108 

PP106 

PC307 

AE204 

PC  307 

OPOOl 

OCOOl 

SI001 

PP110 

3 

0602 

AE103 

AE103 

AE206 

PP106 

PC307 

AE206 

PC  307 

SI004 

SI004 

SF001 

PE  204 

4 

7565 

AE103 

PP101 

AE202 

PP106 

PC307 

AE204 

PC504 

SV002 

SI003 

SF002 

PE  204 

5 

49A 

PP103 

OP004 

PC102 

PP106 

PP205 

PC504 

PC404 

OP002 

OP003 

SI004 

PE206 

6 

21B/49 

AE202 

AV001 

PC106 

PP106 

SI001 

AE204 

PP205 

OR007 

OP005 

OP006 

OR004 

7 

1810 

PP103 

PP101 

PC106 

PP106 

PC307 

AE103 

AE201 

OCOOl 

51004 

SI001 

PE  204 

8 

1110 

AE103 

OP004 

PC108 

AE204 

PC308 

AE104 

PC  308 

SI004 

SF001 

SF002 

PE  204 

9 

1110 

5V001 

AE103 

AV001 

AE206 

OPOOl 

AE206 

OPOOl 

OP002 

OP005 

OP002 

OR004 

10 

1110 

SV002 

OP004 

AE103 

PP108 

PC308 

AE206 

PC  307 

SI001 

OP002 

OP006 

PE  204 

11 

1110 

AE103 

PP103 

PC106 

PP106 

PC308 

AE206 

OCOOl 

OCOOl 

OCOOl 

OP005 

PE  204 

12 

1110 

AV001 

AE202 

AE202 

AE204 

PC307 

PP111 

PP205 

SI004 

SF002 

SI001 

PE204 

13 

1110 

AV003 

PP101 

AE206 

PP106 

PC305 

AE104 

PC  307 

OP002 

OP003 

OP006 

PE  204 

14 

1110 

AE202 

PP102 

SF001 

PP106 

PP206 

AE104 

PC  307 

SF001 

OP002 

OP006 

PE  207 

HFACS-M 

Findings 

Number 

Desig/MOS/AFSC 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

1 

1317 

AE103 

AE201 

AE201 

PP106 

PC306 

AE104 

PC  306 

SI004 

SF002 

SI001 

PE203 

2 

1810 

AE103 

PP112 

PC106 

PP106 

PC304 

AE204 

PC  306 

SP201 

OP006 

OP002 

PE203 

3 

0602 

AV002 

SI001 

PC106 

AE204 

PC304 

PC101 

PP205 

OPOOl 

OP002 

OP006 

OR003 

4 

7523 

AE201 

AE103 

PP108 

AE206 

PC307 

AE204 

PC  306 

PE204 

PE202 

SI001 

OR004 

5 

1310 

AE103 

PP101 

AE202 

PP106 

PC306 

AE206 

PC306 

OP003 

SI004 

SI006 

OR004 

6 

1120 

AV001 

PP102 

AE203 

PP106 

OPOOl 

AE206 

PC  505 

OPOOl 

OR007 

OP006 

PE206 

7 

19A 

AV001 

PP102 

PP101 

PP106 

PC307 

AE204 

PC306 

SF001 

OP006 

SV002 

PE201 

8 

1110 

AV001 

PC206 

PP103 

PP106 

PC304 

AE204 

PC  307 

PC 307 

OP002 

OCOOl 

PE201 

9 

1110 

AE103 

AE102 

PC108 

PP106 

PC307 

PP105 

PC  307 

OPOOl 

OP002 

OP006 

PE203 

10 

1110 

AE103 

PP101 

PC108 

PP106 

PC306 

AE104 

PC510 

SI004 

SF002 

SI001 

PE  208 

11 

1110 

AV001 

PC405 

PC102 

PP106 

PC307 

AE204 

PC  306 

OP003 

OP005 

OCOOl 

OR004 

12 

1110 

AE103 

AE206 

AE206 

AE204 

OPOOl 

AE204 

PC  306 

OP003 

OCOOl 

OP007 

PE203 

13 

1110 

AE103 

PP101 

PC108 

PP106 

PC306 

AE204 

PC  306 

OP002 

OP005 

SI004 

OP006 

14 

1110 

PC306 

PP101 

PP108 

PP106 

PC306 

AE204 

PC  307 

OCOOl 

SP007 

SI001 

OR004 
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B.  SUB  CODE 


DOD  HFACS 

Findings 

Number 

Desig/MOS/AFSC 

i 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

i 

49/15 

A-V 

P-CCPF 

A-JDME 

P-CCPF 

P-APS 

A-SB 

P-APS 

O-P 

O-P 

O-P 

P-TE 

2 

1310 

P-CCPF 

P-CCPF 

P-AF 

P-CCPF 

P-APS 

A-JDME 

P-APS 

O-P 

O-C 

S-IS 

P-CCPF 

3 

0602 

A-S8 

A-SB 

A-JDME 

P-CCPF 

P-APS 

A-JDME 

P-APS 

S-IS 

S-IS 

S-FCKP 

P-TE 

4 

7565 

A-SB 

P-CCPF 

A-JDME 

P-CCPF 

P-APS 

A-JDME 

P-PF 

S-SV 

S-IS 

S-FCKP 

P-TE 

5 

49A 

P-CCPF 

O-P 

P-AF 

P-CCPF 

P-SIS 

P-PF 

P-PMl 

O-P 

O-P 

S-IS 

P-TE 

6 

216/49 

A-JDME 

A-V 

P-AF 

P-CCPF 

S-IS 

A-JDME 

P-SIS 

O-RAM 

O-P 

O-P 

O-RAM 

7 

1810 

A-S8 

P-CCPF 

P-AF 

P-CCPF 

P-APS 

A-SB 

A-JDME 

O-C 

S-IS 

S-IS 

P-TE 

8 

1110 

A-S8 

O-P 

P-AF 

A-JDME 

P-APS 

A-SB 

P-APS 

S-IS 

S-FCKP 

S-FCKP 

P-TE 

9 

1110 

S-SV 

A-SB 

A-V 

A-JDME 

O-P 

A-JDME 

O-P 

O-P 

O-P 

O-P 

O-RAM 

10 

1110 

S-SV 

O-P 

A-SB 

P-CCPF 

P-APS 

A-JDME 

P-APS 

S-IS 

O-P 

O-P 

P-TE 

11 

1110 

A-SB 

P-CCPF 

P-AF 

P-CCPF 

P-APS 

A-JDME 

o-c 

o-c 

O-C 

O-P 

P-TE 

12 

1110 

A-V 

A-JDME 

A-JDME 

A-JDME 

P-APS 

P-CCPF 

P-SIS 

S-IS 

S-FCKP 

S-IS 

P-TE 

13 

1110 

A-V 

P-CCPF 

A-JDME 

P-CCPF 

P-APS 

A-SB 

P-APS 

O-P 

O-P 

O-P 

P-TE 

14 

1110 

A-JDME 

P-CCPF 

S-FCKP 

P-CCPF 

P-SIS 

A-SB 

P-APS 

S-FCKP 

O-P 

O-P 

P-TE 

HFACS-M 

Findings 

Number 

Desig/MOS/AFSC 

i 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

1 

1317 

A-SB 

A-JDME 

A-JDME 

P-CCPF 

P-APS 

A-SB 

P-APS 

C-IS 

C-FCKP 

C-IS 

P-TE 

2 

1810 

A-SB 

P-CCPF 

P-AF 

P-CCPF 

P-APS 

A-JDME 

P-APS 

C-PIO 

O-P 

O-P 

P-TE 

3 

0602 

A-V 

C-IS 

P-AF 

A-JDME 

P-APS 

P-AF 

P-SIS 

O-P 

O-P 

O-P 

O-RAM 

4 

7523 

A-JDME 

A-SB 

P-CCPF 

A-JDME 

P-APS 

A-JDME 

P-APS 

P-TE 

P-TE 

C-IS 

O-RAM 

5 

1310 

A-SB 

P-CCPF 

A-JDME 

P-CCPF 

P-APS 

A-JDME 

P-APS 

O-P 

C-IS 

C-IS 

O-RAM 

6 

1120 

A-V 

P-CCPF 

A-JDME 

P-CCPF 

O-P 

A-JDME 

P-PF 

O-P 

O-RAM 

O-P 

P-TE 

7 

19A 

A-V 

P-CCPF 

P-CCPF 

P-CCPF 

P-APS 

A-JDME 

P-APS 

C-FCKP 

O-P 

c-sv 

P-TE 

8 

1110 

A-V 

P-P8F 

P-CCPF 

P-CCPF 

P-APS 

A-JDME 

P-APS 

O-P 

O-P 

o-c 

P-TE 

9 

1110 

A-SB 

A-SB 

P-AF 

P-CCPF 

P-APS 

P-CCPF 

P-APS 

O-P 

O-P 

O-P 

P-TE 

10 

1110 

A-SB 

P-CCPF 

P-AF 

P-CCPF 

P-APS 

A-SB 

P-PF 

C-IS 

C-FCKP 

C-IS 

P-TE 

11 

1110 

A-V 

P-PML 

P-AF 

P-CCPF 

P-APS 

A-JDME 

P-APS 

O-P 

O-P 

o-c 

O-RAM 

12 

1110 

A-SB 

A-JDME 

A-JDME 

A-JDME 

O-P 

A-JDME 

P-APS 

O-P 

O-C 

O-P 

P-TE 

13 

1110 

A-SB 

P-CCPF 

P-AF 

P-CCPF 

P-APS 

A-JDME 

P-APS 

O-P 

O-P 

C-IS 

O-P 

14 

1110 

P-APS 

P-CCPF 

P-CCPF 

P-CCPF 

P-APS 

A-JDME 

P-APS 

O-C 

C-MPT 

C-IS 

O-RAM 
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C.  DOD  HFACS  SUB 


ASB  |  A  IDMf  |  APf  |  AV  |  P  Pt  |  P  T{  |  P  StS  |  PCCPt  |  P  AF  |  P  PMt  |  P  PF  |  P  PBF  |  P  APS  |  SIS  |SFCKP|SPK>|  S  SV  |  O  RAM  |  PC 


■I  ;l  °l 


“I  'I  °l  »l  ’I  >T~ 


0.123  0.000  0.032  0.000  0.071  0.026  0.143  0.039  0.006  0.013  0  000  0.110  0  0/8  0.04  5  0.000  0.019  0.019  0.032  0.156 


Pbar  = 
PcBar  = 
Kappa  = 


0.352 

0.096 

0.281 


A  SB  |  A  IDMt  |  A  Pt  |  A  V  |  P  Pf  |  PH  |  P  SIS  [  P  Cf  Pf  |  P  Af  |  P  PMt  |  P  PF  |  P  PBf  |  P  APS  |  5  IS  |  S  fCKP  |  S  PIO  |  S  SV  |  O  RAmT" 


OOP  SWO  SUB 


-'l— -;±- -i-jL. 


j-.J 


0.091  0.130  0.000  0  039  0  000  0.078  0.026  0.104  0.026  0.000  0  000  0  000  0.117  0.052  0.065  0.000  0.026  0.013  0.039  0.195 


Pbar  = 

Pefiar  = 
Kappa  = 


0.329 

0.106 

0.250 


DOD  NON  SWO  SUB 

A  SB  |  A  IDMt  | 

A  Pt  | 

AAV  | 

P  PE  | 

PTI  | 

P  SIS  | 

P-CCPf  1 

P  At  | 

P  PMt  | 

P  Pf  | 

P  PBt  | 

P  APS  | 

SIS  Is  FCKP  | 

S-PK)  I 

ssv  | 

o  ram| 

oc  I 

OP 

Pt 

1 

3  1 

0 

1 

0 

0 

0 

2 

0 

0 

0 

0 

0 

0  0 

0 

0 

0 

0 

0 

7 

2 

I  0 

0 

1 

0 

0 

0 

4 

0 

0 

0 

0 

0 

0  0 

0 

0 

0 

0 

1 

7 

3 

0  3 

0 

0 

0 

0 

0 

0 

4 

0 

0 

0 

0 

0  0 

0 

0 

0 

0 

0 

0.429 

7 

4 

0  0 

0 

0 

0 

0 

0 

7 

0 

0 

0 

0 

0 

0  0 

0 

0 

0 

0 

0 

7 

5 

0  0 

0 

0 

0 

0 

1 

0 

0 

0 

0 

0 

5 

1  0 

0 

0 

0 

0 

0 

7 

6 

2  4 

0 

0 

0 

0 

0 

0 

0 

0 

1 

0 

0 

0  0 

0 

0 

0 

0 

0 

7 

7 

0  1 

0 

0 

0 

0 

1 

0 

0 

1 

1 

0 

3 

0  0 

0 

0 

0 

0 

0 

0.143 

7 

8 

0  0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

1  0 

0 

1 

1 

1 

3 

0.143 

7 

9 

0  0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

3  0 

0 

0 

0 

1 

3 

0286 

7 

10 

0  0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

3  2 

0 

0 

0 

0 

2 

0238 

7 

11 

0  0 

0 

0 

0 

5 

0 

1 

0 

0 

0 

0 

0 

0  0 

0 

0 

1 

0 

0 

04/6 

7 

Total 

6|  9| 

°l 

A 

°l 

A 

*1 

14| 

4I 

A 

*1 

°l 

*\ 

8|  >1 

°l 

>1 

>1 

>1 

9 

4  000 

77 

Pi 

0.0/8  0.117 

0  000 

0026 

0000 

0.065 

0.026 

0  182 

0.052 

0013 

0026 

0000 

0.104 

0  104  0  026 

0000 

0.013 

0026 

0026 

0.11/ 

Pbar 

PeBar 

Kappa 


0.364 

0.099 

0.293 
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D.  HFACS-M  SUB 


Kappa «  0.289 
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E 


CATEGORICAL 


DOD  HFACS 

F 

'ridings 

Number 

Desig/MOS/AFSC 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

1 

49/15 

A 

P 

A 

P 

P 

A 

P 

0 

0 

0 

P 

2 

1310 

P 

P 

P 

P 

P 

A 

P 

0 

0 

0 

P 

3 

0602 

A 

A 

A 

P 

P 

A 

P 

0 

0 

0 

P 

4 

7565 

A 

P 

A 

P 

P 

A 

P 

S 

S 

s 

P 

5 

49A 

P 

0 

P 

P 

P 

P 

P 

0 

0 

s 

P 

6 

21B/49 

A 

A 

P 

P 

S 

A 

P 

0 

0 

0 

0 

7 

1810 

A 

P 

P 

P 

P 

A 

A 

0 

5 

s 

P 

8 

1110 

A 

0 

P 

A 

P 

A 

P 

S 

S 

s 

p 

9 

1110 

S 

A 

A 

A 

0 

A 

0 

0 

0 

0 

0 

10 

1110 

5 

0 

A 

P 

P 

A 

P 

S 

0 

0 

p 

11 

1110 

A 

P 

P 

P 

P 

A 

0 

0 

0 

0 

p 

12 

1110 

A 

A 

A 

A 

P 

P 

P 

S 

S 

s 

p 

13 

1110 

A 

P 

A 

P 

P 

A 

P 

0 

0 

0 

p 

14 

1110 

A 

P 

S 

P 

P 

A 

P 

S 

0 

0 

p 

HFACS-M 

F 

'ndings 

Number 

Desig/MOS/AFSC 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

1 

1317 

A 

A 

A 

P 

P 

A 

P 

C 

C 

c 

P 

2 

1810 

A 

A 

P 

P 

P 

A 

P 

C 

0 

0 

P 

3 

0602 

A 

C 

P 

A 

P 

P 

P 

0 

0 

0 

0 

4 

7523 

A 

A 

P 

A 

P 

A 

P 

P 

P 

c 

0 

5 

1310 

A 

P 

A 

P 

P 

A 

P 

0 

C 

c 

0 

6 

1120 

A 

P 

A 

P 

0 

A 

P 

0 

0 

0 

P 

7 

19A 

A 

P 

P 

P 

P 

A 

P 

C 

0 

c 

p 

8 

1110 

A 

P 

P 

P 

P 

A 

P 

0 

0 

0 

p 

9 

1110 

A 

A 

P 

P 

P 

P 

P 

0 

0 

0 

p 

10 

1110 

A 

A 

P 

P 

P 

A 

P 

C 

c 

c 

p 

11 

1110 

A 

P 

P 

P 

P 

A 

P 

0 

0 

0 

0 

12 

1110 

A 

A 

A 

A 

0 

A 

P 

0 

0 

0 

p 

13 

1110 

A 

A 

P 

P 

P 

A 

P 

0 

0 

c 

0 

14 

1110 

P 

P 

P 

P 

P 

A 

P 

0 

c 

c 

0 
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F 


DOD  HFACS  CATA 


DOD  CAT 

0 

S 

P 

A 

Pi 

1 

0 

2 

2 

10 

0.516 

2 

3 

0 

7 

4 

0.330 

3 

0 

1 

6 

7 

0.396 

4 

0 

0 

11 

3 

0.637 

5 

1 

1 

12 

0 

0.725 

6 

0 

0 

2 

12 

0.736 

7 

2 

0 

11 

1 

0.615 

8 

9 

5 

0 

0 

0.505 

9 

10 

4 

0 

0 

0.560 

10 

9 

5 

0 

0 

0.505 

11 

2 

0 

12 

0 

0.736 

Total 

36 

18 

63 

37 

6.264 

Pi 

0.234 

0.117 

0.409 

0.240 

Pbar  = 

0.569 

PeBar  = 

0.293 

Kappa  = 

0.391 

DD  swo  a 

O 

S 

P 

A 

Pi 

1 

0 

2 

0 

5 

0.524 

2 

2 

0 

3 

2 

0.238 

3 

0 

1 

2 

4 

0.333 

4 

0 

0 

4 

3 

0.429 

5 

1 

0 

6 

0 

0.714 

6 

0 

0 

1 

6 

0.714 

7 

2 

0 

5 

0 

0.524 

8 

3 

4 

0 

0 

0.429 

9 

5 

2 

0 

0 

0.524 

10 

5 

2 

0 

0 

0.524 

11 

1 

0 

6 

0 

0.714 

Total 

19 

11 

27 

20 

5.667 

PJ 

0.247 

0.143 

0.351 

0.260 

Pbar  = 

0.515 

PeBar  = 

0.272 

Kappa  = 

0.334 

NON-SWO 

O 

S 

P 

A 

Pi 

1 

0 

0 

2 

5 

0.524 

2 

1 

0 

4 

2 

0.333 

3 

0 

0 

4 

3 

0.429 

4 

0 

0 

7 

0 

1.000 

5 

0 

1 

6 

0 

0.714 

6 

0 

0 

1 

6 

0.714 

7 

0 

0 

6 

1 

0.714 

8 

6 

1 

0 

0 

0.714 

9 

5 

2 

0 

0 

0.524 

10 

4 

3 

0 

0 

0.429 

11 

1 

0 

6 

0 

0.714 

Total 

17 

7 

36 

17 

6.810 

Pj 

0.221 

0.091 

0.468 

0.221 

Pbar  =  0.619 

PeBar  =  0.324 

Kappa  =  0.436 
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G 


HFACS-M  CATA 


M  CAT 

0 

C 

P 

A 

Pi 

1 

0 

0 

1 

13 

0.857 

2 

0 

1 

6 

7 

0.396 

3 

0 

0 

10 

4 

0.560 

4 

0 

0 

11 

3 

0.637 

5 

2 

0 

12 

0 

0.736 

6 

0 

0 

2 

12 

0.736 

7 

0 

0 

14 

0 

1.000 

8 

9 

4 

1 

0 

0.462 

9 

9 

4 

1 

0 

0.462 

10 

7 

7 

0 

0 

0.462 

11 

6 

0 

8 

0 

0.473 

Total 

33 

16 

66 

39 

6.780 

Pj 

0.214 

0.104 

0.429 

Pbar=  0.616 

Pe8ar=  0.305 

Kappa  =  0.448 

M  SWO  CAT 

O 

C 

P 

A 

Pi 

1 

0 

0 

1 

6 

0.714 

2 

0 

0 

3 

4 

0.429 

3 

0 

0 

6 

1 

0.714 

4 

0 

0 

6 

1 

0.714 

5 

1 

0 

6 

0 

0.714 

6 

0 

0 

1 

6 

0.714 

7 

0 

0 

7 

0 

1.000 

8 

6 

1 

0 

0 

0.714 

9 

5 

2 

0 

0 

0.524 

10 

4 

3 

0 

0 

0.429 

11 

3 

0 

4 

0 

0.429 

Total 

19 

6 

34 

18 

7.095 

Pj  0.247  0.078  0.442 

Pbar=  0.645 

PeBar  =  0.317 

Kappa  =  0.481 

M  NON-SWOCAT 

O 

C 

P 

A 

Pi 

1 

0 

0 

0 

7 

1.000 

2 

0 

1 

3 

3 

0.286 

3 

0 

0 

4 

3 

0.429 

4 

0 

0 

5 

2 

0.524 

5 

1 

0 

6 

0 

0.714 

6 

0 

0 

1 

6 

0.714 

7 

0 

0 

7 

0 

1.000 

8 

3 

3 

1 

0 

0.286 

9 

4 

2 

1 

0 

0.333 

10 

3 

4 

0 

0 

0.429 

11 

3 

0 

4 

0 

0.429 

Total 

14 

10 

32 

21 

6.143 

Pj 

0.182 

0.130 

0.416 

0.273 

Pbar  =  0.558 

PeBar  =  0.297 

Kappa  =  0.372 
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H. 


OVERALL  ANALYSIS 


DOD  HFACS 

HFACS-M 

swo 

7 

7 

NON-SWO 

7 

7 

Overall 

SWO 

NON-SWO 

DOD  HFACS  (Nano-code) 

0.154 

0.114 

0.198 

HFACS-M  (Nano-code) 

0.182 

0.202 

0.137 

DOD  HFACS  (Sub-code) 

0.281 

0.25 

0.293 

HFACS-M  (Sub-code) 

0.355 

0.362 

0.289 

DOD  HFACS  (Category) 

0.391 

0.334 

0.436 

HFACS-M  (Category) 

0.448 

0.481 

0.372 
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